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ABSTRACT 



This document consists of course materials for adult 
participants in a graduate- level course in "Survey of Human Resource 
Development." Following an introductory section on training materials, 
including a course pretest, and an introduction to graduate study, the course 
covers the following topics: (1) history of training; (2) behavioral sciences 

in training and development; (3) organization and management of training; (4) 
selection and development of the training staff; (5) determining training 
needs; (6) the performance audit; (7) workforce planning; (8) instructional 
systems; (9) using external resources; (10) training costs and benefits; and 
(11) legal aspects of training. Three appendixes contain answers to the 
course pre-test, progress tests, and the course final exam. Course materials 
include information sheets, activities, tips and suggestions, outlines, 
multiple-choice tests, glossaries, and charts and tables. (KC) 
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INTRODUCTIONS 



In any college course, one of the most difficult, yet important, steps is to become 
acquainted with the instructor and other participants. To make the process of introducing 
yourself a little easier, consider the following statements. There is space for you to jot 
down some thoughts and ideas. It's fair to look at this later when you are addressing the 
group. 



1 . Who am I? (name, etc.) 

2. What I like most about my job is: 



3. What motivates me is: 
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SS# Date 

Human Resource Development Department The University of Tennessee 

Pre-test 



Directions: Each of the following items contains four (4) alternatives, one of which provides 

the best answer to the item. Circle the letter which precedes the best answer. 

1. The program to train shipyard workers during World War I was based on the principle of 

A. classroom training separated from the worksite. 

B. avoiding the use of work supervisors as trainers. 

C. traditional long-term apprenticeship training. 

D. show, tell, do and check. 

2. A manager can conclude that a formal training program will be useful when 

A. problems exist that can be partly or wholly solved by giving personnel identifiable 
items of knowledge, skill or attitude. 

B. individuals or groups of workers are not performing well enough to meet job 
standards set for them. 

C. the organization is not meeting the objectives included in annual or long-term plans. 

D. some or all jobs in the organization require specific knowledge and skill for successful 
performance. 

3. The most important function of a basic training policy is to 

A. describe the specific training programs that will be carried out. 

B. show how training will be used to help achieve the goals of the organization. 

C. enumerate the problems that have resulted in the establishment of a training program. 

D. demonstrate to top management that a training program is necessary. 

4. A recommended procedure for achieving cost-effective training programs is to 

A. determine the absolute minimum training needs and design low-cost programs to meet 
these. 

B. design and estimate costs for an optimum program and compare alternatives with this. 

C. always use the lowest-cost training method available. 

D. concentrate on training costs since benefits cannot be measured in financial terms. 

5. Why might a training manager choose a rental-deferred purchase plan to buy an expensive 
videotape machine? 

A. Rental-purchase plans are almost always less expensive than outright purchase. 

B. The plan might allow the purchase to be included in the operating budget rather than 
in a tight capital budget. 

C. The plan would always place the purchase in the capital budget rather than in the 
operating budget. 

D. Such purchases need not be charged to the costs of training. 
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6. One of the most important findings of the Hawthorne studies for Western Electric was the 
conclusion that 

A. physical and environmental influences have little effect on worker efficiency. 

B. experimental methods are of little value in improving work effectiveness. 

C. social interactions and group influences may have a stronger effect on work efficiency 
than physical factors in the environment. 

D. the best incentive for increased productivity is higher wages. 

7. Abraham H. Maslow’s hierarchy of needs would predict that a worker whose 
physiological, safety, belonging and love needs were satisfied would be motivated 

A. largely by the incentive of salary increases. 

B. by chances to satisfy esteem and self-actualization needs. 

C. by further chances to satisfy physiological, safety, belonging and love needs. 

D. only by the fear of punishment or loss of reward. 

8. According to Frederick Herzberg’s two factor theory, which of the following lists include 
job factors that are likely to be strong motivators and sources of satisfaction? 

A. achievement, responsibility and the work itself. 

B. working conditions and salary. 

C. status and job security. 

D. interpersonal relations and supervision. 

9. The Managerial Grid was developed by Robert R. Blake and Jane S. Mouton to help 
demonstrate that 



A. managers concentrate too much on production and do not show enough concern for 
people. 

B. organizational development should rely on graphic methods. 

C. people and production are complementary and mutually reinforcing in reaching goals. 

D. if managers concentrate on the people in an organization, production goals will take 
care of themselves. 

10. Well-planned training facilities directly affect the success of training programs by 

A. reducing the need for visual display material. 

B. facilitating successful lectures. 

C. making up for deficiencies in the instructional materials. 

D. creating the kinds of attitudes that contribute to learning. 




11. Training that is conducted outside the employee’s normal work hours, attended 

voluntarily, not directly related to the employee’s job, and in which the employee does no 
production work is 

A. normally not compensable time and does not contribute to the hours in a standard 
workday or workweek. 

B. by law always subject to overtime pay. 

C. specifically included in federal wage and hour legislation as compensable time. 

D. not allowed for firms working on federal contracts. 
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12. Which of the following would most likely qualify as true apprenticeship training eligible 
for government guidance and assistance? 

A. A one-year work/study program to train clerk- typists. 

B. An on-the-job sales training program with no classroom instruction. 

C. A work experience training program to help 14- to 16-year-old migrants learn to do 
farm work. 

D. A three-year program combining classroom instruction and on-the-job experience to 
train bricklayers. 

13. The Equal Employment Opportunity Commission’s guidelines for selecting job candidates 

A. prohibit the use of employee selection tests. 

B. view all pencil-and-paper tests as being fairer than other selection methods. 

C. do not provide guidance in the use of employment selection tests. 

D. allow the use of selection tests if the tests have been properly validated. 

14. The presentation portion of the lesson plan format 

A. lets the learners practice what they have observed. 

B. demonstrates the skills to be learned. 

C. measures the achievement of the learner. 

D. should sustain the attention of the learner. 

15. The National Labor Relations Act became law 

A. during World War II. 

B. just after the Depression. 

C. during the Korean War. 

D. during the Vietnam War. 

16. When interviewing for a needs assessment, you should 

A. use a variety of questions. 

B. set no time limit. 

C. try to avoid taking notes during the interview. 

D. know something about the worker being interviewed. 
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Human Resource Development Department 501 The University of Tennessee 

Survey of Human Resource Development Knoxville, Tennessee 



COURSE OUTLINE/SYLLABUS 



1. Course Description : (from Graduate Catalog) 

Training and development as practiced in organizations: needs assessment, transfer of 
workplace skills, evaluation, development of training program proposals, assessment of 
personal competencies, values, goals, and training program design and administration. 

2. Anticipated Participants ; 

This course is designed for individuals in (or preparing for) all levels of the training and 
performance improvement community who are responsible for developing, 
implementing, teaching, evaluating, or managing educational/training courses and 
programs related to successful job performance and career mobility. 

3. Course Objectives ; 

3.1 Help the participant become a more skillful, confident, and resourceful trainer 

3.2 Review and clarify strategies for meeting training and development needs 

3.3 Discuss the role and function of a trainer 

• Guiding a group of specialists 

• Goals and objectives for the training department 

• Training costs and benefits 

• Making the training department important to management 

3.4 Provide some practical understanding of selected topics from item 4 

3.5 Initiate a selected reading program 

4. Subject Matter Topics : 



4. 1 How to: Read/Write/Make a Speech/Use a Library Information Sheets 

4.2 The Value of Films and Video Tapes Information Sheet 

4.3 Audiovisual Media Evaluation Form Assignment Sheet 

4.4 Tips for Successful Writing Information Sheet 

4.5 History of Training Progress test No. 1 

4.6 Traditional Beliefs Based on Wisdom and Common Sense . . . Information Sheet 

4.7 Lesson Plan Format Information Sheet 

4.8 Behavioral Sciences in Training and Development Progress test No. 2 

4.9 Scientific Management Information Sheet 

4.10 Peer Training and Teamwork Information Sheet 

4.11 Effective and Ineffective Teams Information Sheet 
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4.12 Example of a Scheduling Work Sheet Information Sheet 

4.13 Team Member — Rating, Evaluation, and Ranking Forms . . . Assignment Sheets 

4.14 Organization and Management of Training Progress test No. 3 

4. 1 5 Workplace Goals that Justify Investing Private Resources 

in Training Information Sheet 

4.16 Workforce Training Information Sheet 

4.1 7 The Impact of a Training Experience Information Sheet 

4.18 Training Organization Charts Information Sheet 

4.19 Training Policy Analysis Information Sheet 

4.20 Selection and Development of the Training Staff Progress test No. 4 

4.21 Competency Analysis for Trainers Information Sheet 

4.22 Determining the Demand for Skilled Workers Module 

4.23 Labor Market Demand Assignment Sheet 

4.24 Determining Training Needs Progress test No. 5 

4.25 Needs Assessment Information Sheet 

4.26 Needs Assessment: A Rationale and Methodology Information Sheet 

4.27 The Performance Audit Progress test No. 6 

4.28 Workforce Planning Progress test No. 7 

4.29 Instructional Systems Progress test No. 8 

4.30 Using External Resources Progress test No. 9 

4.31 The Self-Paced Learning Environment Information Sheet 

4.32 Individualization of Instruction Information Sheet 

4.33 Programmed Instruction Information Sheet 

4.34 Research on Preparing Self-Instructional Materials Information Sheet 

4.35 A Point of View on Individualized Instruction Information Sheet 

4.36 Training Costs and Benefits Progress test No. 10 

4.37 Determining the Cost Effectiveness of Training Module 

4.38 An Accounting Framework for Investments in HRD Information Sheet 

4.39 Legal Aspects of Training Progress test No. 11 

4.40 Checklist for Presentation Skills Job Performance Aid 

4.41 Technology Transfer: Model Programs in Europe Information Sheet 



5. Instructional Format : 

A blend of instructional strategies, including lecture-discussion, individual and group 
activities, self-learning modules, supplemental reading, video tapes, individual 
consultation, and written assignments will be utilized. Participants are expected to be 
active and, hopefully, enthusiastic as they work and share with others in the learning 
process. A strong accent is placed on fostering interaction to engage participants in 
pooling their abilities and experiences. Project assignments, and the combination of 
theory with practice, provide for full immersion in the subject matter to make training 
optimally effective. Prescriptive feedback, including suggestions for improvement, is 
provided on all course products submitted. 
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6. Assignments : (See Assignment Sheet No. 1, titled COURSE DELIVERABLES, for 
details) 

Paper and pencil tests (pre-test and post-test including essay items) as well as course 
activities and products (deliverables) are used in determining final grades. All products 
shall be prepared and submitted in accordance with requirements and instructional 
materials used in the course. Products will be critiqued, graded, and returned to the 
developer(s). 

Course participants shall: 

6.1 Preview audiovisual media dealing with training and submit evaluation forms. 

6.2 Submit duplicated documents: clippings, reprints, or photo copies of carefully 
selected relevant text. Submittals are to be made during the first part of the course 
term so that selected items can be shared with the class through "mini-briefs." 

6.3 Attend a meeting dealing with training or a closely related topic and submit a one- 
page typed report. 

6.4 Identify and describe the salient characteristics of two labor market signaling 
approaches. 

6.5 Write a scholarly paper. Select subject/topic from list on Assignment Sheet No. 1 
and follow the APA Publication Manual (4 th edition). 

6.6 Upgrade the course glossary by (a) making beneficial changes to existing entries 
on course glossary and glossary of terms in textbook, and (b) adding new words 
and terms relevant to the course. Credit sources of changes and additions. 

7. Understanding Grading Practices : 

Grading constitutes a complex and difficult process. While human beings cannot be 
pigeonholed, they can be judged on the basis of their performance. The following 
descriptive profiles attempt to explain why different participants obtain different final 
grades. 

The "A" Grade — Superior Performance: Traditionally, this highest grade 
represents some form of extra effort and superior performance. 

"A" participants are punctual and present for the entire class period. They have virtually 
perfect attendance. Their commitment to the course resembles that of the instructor. 
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"A" participants are prepared for class. They always read, thoroughly and critically, the 
instructional materials. Their attention to detail is such that they submit all work on time 
and occasionally submit suggestions for improvements to the instructional materials 
being used. 

"A" participants show interest in the class and in the subject. They take responsibility 
for their own learning and look up or dig out what they don’t understand. They 
participate in class, asking relevant questions and making thoughtful comments. 

"A" participants bring a background with them to class and share relevant experiences 
with others. 

"A" participants have retentive minds. They are able to connect past learning with the 
present. They apply facts, principles, techniques, theories, concepts, and procedures 
previously learned to new learning situations. 

"A" participants have a winning attitude. They have both the determination and the self- 
discipline necessary for success. They show initiative, doing things they have not been 
told to do and going the "extra mile" to ensure success. 

"A" participants have something special. It may be exceptional intelligence and insight. 

It may be unusual creativity, organizational skills, commitment, perseverance — or a 
combination thereof. These gifts are evident to the instructor and to the other participants 
as well. 

"A" participants make high grades on examinations and course submittals. 

The "C" Grade — Performance Below the Standard Expected of Graduate Students 

"C" participants miss class, arrive late, and/or leave early. They put other priorities ahead 
of academic work. In some cases, their health or fatigue renders them physically unable 
to keep up with the demands of high-level graduate performance. 

"C" participants prepare their assignments, but in a perfunctory manner. Their work may 
be sloppy or careless. There is little or no evidence that quality control checks were made 
before an assignment was submitted. At times, submittals are incomplete and/or late. 

"C" participants are not visibly committed to the class. They participate without 
enthusiasm. Their body language often expresses boredom or worse. 

"C" participants vary in talent. Some are academically talented but show undeniable 
signs of poor self-management or bad attitudes. Others are diligent but simply average in 
academic ability. 
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"C" participants obtain mediocre or inconsistent results on course submittals. They have 
some concept of what is going on but clearly have not mastered the material or method. 

Note . These descriptive profiles were adapted from "Clarifying Grade Expectations" by J. H. 
Williams, 1993, The Teaching Professor . August/September, p. 1. 

8. References : 

8.1 Campbell, C. P. (1997, February). Determining the demand for skilled workers. 2 

8.2 Campbell, C. P. (Ed.). (1996V Education and training for work: Planning 
programs. Lancaster, PA: Technomic. 1 

8.3 Campbell, C. P. (1994, March). Determining the cost effectiveness of training. 2 

Footnotes 

'Course textbook, available from UT bookstore and the publisher. (The author/instructor 
receives no royalties (profit) from the sale of this work.) 

2 Module, available from the Human Resource Development Department, The University of 
Tennessee, Knoxville, TN 37996-1900 

9. Notes: 



9. 1 In the event that some unforeseen event or circumstance occurs, during the course 
term, that adversely affects your performance, e.g., a death in the family or 
personal illness, please advise the instructor in writing so that proper 
consideration can be given. 

9.2 The language of instruction is the American form of English. 

9.3 In post-secondary settings, it is the student’s responsibility to request an 
accommodation for a documented disability. Students with a disability, according 
to the Rehabilitation Act of 1973 and the Americans with Disabilities Act of 
1990, should feel free to contact the Office of Disability Services, 191 Hoskins 
Library, 974-6087, for assistance. 

9.4 In response to previous student concerns, and as a matter of courtesy, portable 
electronic communication devices with audible signals are to be deactivated 
(turned off) during class. 

9.5 Slipshod and untidy work is not acceptable. Don’t let the quality and 
completeness of any submittal fall below your personal standards or those of the 
course. Details receive close attention. A quality control (QC) check by you and 
others is recommended before every course submittal. 
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Human Resource Development Department 501 The University of Tennessee 

Survey of Human Resource Development Assignment Sheet No. 1 





COURSE DELIVERABLES 


Charge: 


To share expertise, knowledge and learning acquired in a selected subject/topic. 


Activity 1. 

Audiovisual 

media 


Preview three (3) films/video tapes/video disks/sound slide 
programs/filmstrips/audio tapes dealing with training principles, issues, 
methodologies, techniques, etc., and submit typed audiovisual media 
evaluation forms. Turn in one (1) evaluation by class session four and the 
other two (2) not later than class session eight. 

Submittals are rated with a (/+), (/), or (/-). A (/+) is the highest 
rating and (/-) the lowest. A quality control check of each audiovisual 
media evaluation form is recommended before its submittal. 


Activity 2. 

Duplicated 

documents 


Submit four (4) different duplicated documents, clippings, reprints or photo 
copies of text discussing any of the subjects/topics listed on the next page 
(Page 2 of 2). Turn in one (1) document by class session three and the other 
three (3) at regular intervals during the first seven sessions of the course 
term so that selected items can be shared with the class through "mini- 
briefs." 

Each submittal needs to be carefully selected from a book, journal, 
magazine, etc. for its quality and relevance to a particular subject/topic. 
Focus on applications/information dealing with print "hard copy" vice 
audiovisual media or methods. Show your comprehension of the content by 

highlighting the text that is important to the subject/topic identified. 

Submittals shall include (a) the source reference in accordance with the 
Publication Manual of the American Psychological Association (APA), 
fourth edition; (b) the appropriate subject/topic and its number from the list 
on Page 2 of 2; and (c) your name and social security number (last 4 
numbers only). Use a cover page of your own design to provide this 
information. 

A judgment will be made concerning the (a) care exhibited in selecting the 
document, (b) relevance and quality, (c) comprehension shown through 
selected highlighting, as well as (d) accuracy of the source reference, and 
(e) completeness of each submittal. Submittals are rated with a (/ +), 

(/), or (/-). A (/+) is the highest rating and (/ -) the lowest. Note: 
Slipshod and untidy work is not acceptable. A quality control check is 
recommended before submitting documents. 


Activity 3. 

Meeting 


Attend a meeting dealing with training or personal matters and submit a one- 
page typed report (in accordance with the APA publication manual) by class 
session 12. Meetings of the Smoky Mountain Chapter, American Society 
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Activity 4. 

Labor market 
signaling 



Activity 5. 

Paper 



Activity 6. 
Glossary 



Identify and describe the salient characteristics of any two (2) of the five 
labor market signaling approaches. Include a table showing their advantages 
and disadvantages. See assignment sheet titled COURSE DELIVERABLE 
— LABOR MARKET DEMAND for complete details. 

Write a scholarly paper, approximately six double-spaced typewritten pages 
in length on any of the subjects/topics listed on Page 2 of 2. It is to be a 
derivative work (based on preexisting works) prepared in accordance with 
the APA Manual. The paper shall include a (a) minimum of one figure or 
table and (b) reference list. Don't forget to use headings, mention Figures 
and/or Tables in your text, and document your work by citing sources. 

Staple pages in top left corner — do not use a binder or plastic page 
protectors. The paper is due not later than class session 13. 

Upgrade the course glossary by (a) making beneficial changes to existing 
entries on course glossary and glossary of terms in textbook and (b) adding 
new words and terms relevant to the course. Credit sources of changes and 
additions. 



Subjects/Topics: The subject/topic for activities 2 and 5 should be selected from the 

following list. Any of these topics or a sub-division of the topic may be 
chosen for activity 5 without instructor approval. If an alternate topic is 
preferred, a brief written proposal must be submitted to the instructor for 
approval prior to any writing effort. Focus on applications/information 
dealing with print "hard copy" vice audiovisual media or methods. 



1. Labor Market Skill Requirements and Training Output 

2. Needs Analysis/ Assessment 

3. Job and/or Task Analysis 

4. Non-formal Vocational Education and Training 

5. Job Performance Measures (Criterion-Referenced Performance Tests) 

6. The Costs and Benefits of Training 

7. Training Program Evaluation (Internal and/or External) 

8. Training and Follow-up Activities (How to Combat Training Failure Which Occurs 
When the Training Experience Is Not Applied to the Work Situation) 

9. The Hawthorne Studies 

10. Scientific Management (Frederick Winslow Taylor) 

1 1 . The Need for and Purposes of Licensing and Certification 
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THE VALUE OF FILMS AND VIDEO TAPES 



More and more trainers are recognizing the value of audiovisual media. As a result, they are 
using films and video tapes to a greater degree in their courses and programs. Studies show that 
a film or video tape can make just about any subject matter 25% more understandable. In 
comparison to print materials, it was estimated that a well-made video tape can increase interest 
in a topic by as much as 40%. Furthermore, researchers found that persons viewing films and 
video tapes retain approximately 33% more information for as long as 1 year after the 
viewing. 

The Chinese proverb, "To see a thing once is better than to hear it a hundred times," emphasizes 
the inadequacy of using only words in communication. People have multiple sensory abilities. 
They can see, feel, taste, smell, and hear. We know that training techniques which stimulate as 
many of these senses as possible enhance learning and strengthen retention. 

Films and video tapes combine two powerful senses — sight and hearing — and support 
training/learning in ways such as the following: 

1 . Difficult concepts can be presented on a practical level in a short period of time. 

2. Because sound and picture are presented simultaneously, attention can be focused on 
the critical elements of a mechanical process or a human interaction. 

3. Because people are involved, viewers often relate directly to the situations depicted 
and see ready applications for the information presented. 

4. They help avoid boredom inherent in lessons which depend on a single instructional 
technique. 

5. They provide a means for the storage and prompt retrieval of important information 
that can be shared with many individuals or groups in exactly the same way. 

6. They set the stage for role play, case study, discussion, and other learning 
opportunities. 

7. Audio can be provided in various languages and volume can be adjusted for those 
with a hearing problem. 

Nevertheless, the matters of cost/availability, hardware requirements, outdated clothing, 
hairstyles, vocabulary, content, etc. can be disadvantages. 
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AUDIOVISUAL MEDIA EVALUATION FORM 



REVIEWER DATE 

REVIEWED FOR 

TITLE 

SUBJECT MATTER 

PRODUCER/DISTRIBUTOR 

DATE PRODUCED LENGTH (TIME) 

RECOMMENDED AGE LEVEL 

□ AUDIO □ SLIDES □ FILM □ FILMSTRIP □ VIDEOTAPE / □ COLOR □ B & W 



RATING 





Poor 


Average 


Excellent 


QUALITY 


1 . Sound quality (audibility/voice/music) 


0 


1 2 


3 


4 


5 


2. Picture quality (clarity/color) 


0 


1 2 


3 


4 


5 


3. Narration/acting 


0 


1 2 


3 


4 


5 


CONTENT AND ORGANIZATION 


1 . Content is current 


0 


1 2 


3 


4 


5 


2. Vocabulary & concepts are appropriate 


0 


1 2 


3 


4 


5 


3. Organized/logically presented 


0 


1 2 


3 


4 


5 


4. Provides interest/appeal 


0 


1 2 


3 


4 


5 


5. Content accurate 


0 


1 2 


3 


4 


5 



SYNOPSIS OF CONTENT 
(Continue on back of sheet if necessary) 



POOR 




ESTIMATE OF OVERALL VALUE OF THE MEDIA 



AVERAGE 



EXCELLENT 

5 



18 



13 



0 



1 



2 



3 



4 
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VIDEO TAPES 



Beyond Theory Y 




14 min. 


Business Behaviorism and the Bottom Line 




20 min 


Collision Course 




48 min. 


Communicating Non-Defensively: Don’t 
Take It Personally 




21 min. 


Conflict on the Line, A Case Study 




15 min. 


Crisis Bargaining 




35 min. 


Dealing With Difficult People 




20 min. 


Discipline Without Punishment 




20 min. 


Doing Business Internationally: The Cross- 
Cultural Challenges 

Effective Uses of Power and Authority 




45 min. 
32 min. 


Everyone Can Be A Winner 
"Team Building" 




22 min. 


Fair Employment Practice 




Part I, 27 min. 
Part II, 14 min. 
Part III, 30 min. 


First Line Supervisor 




Module 5 


Group Dynamics, Groupthink 




27 min. 


Guide to Business Negotiating 


Roger Dawson 


70 min. 


How To Be A No Limit Person 


Dr. Wayne Dyer 


57 min. 


If Japan Can . . . Why Can’t we? 




77 min. 


In Search of Excellence 




88 min. 
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Introduction to Supervision, Supervision 
Series 

Job Orientation — The Supervisor's 
Role in Training New Employees 

Megatrends 

Modern Times: Effects of Production Line on 
Workers 

A New Look At Motivation 

The One Minute Manager 

The Principles and Practices of MBO 

Problem Solving: A Process for Managers 

Productivity and the Self-fulfilling Prophesy: 
The Pygmalion Effect 

The Self-motivated Achiever 

Strategy for Productive Behavior 

Team Building 

Team Building (2 nd Edition) 

Team Building: Making the Task the Boss 

Teamworking 

Theory X and Theory Y : The World of 
Douglas McGregor 

Time of Your Life 

Time Management System 

Type Z: An Alternative Management Style 

Up the Organization 



18 min. 

Charles Chaplin 87 min. 

32 min. 
50 min. 

20 min. 

« 

29 min. 

30 min. 
30 min. 
30 min. 

19 min. 

18 min. 

13 min. 

Part I, 30 min. 
Part II, 30 min. 

27 min. 

Part I, 50 min. 
Part II, 50 min. 

26 min. 
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W. Edwards Deming Lecture 




Part I, 78 min. 
Part II, 78 min. 


The Winner’s Attitude 


Zig Ziglar 


41 min. 


Working Together Works 




28 min. 


Working with Difficult People 




25 min. 



"Frederick Taylor and Scientific Management" 
Approximately $475 purchase, $150 rental 



Distributed by 

Salenger Films, Inc. 

1635 12th Street 
Santa Monica, CA 90404 
Phone (800) 775-5025 
(310) 450-1300, 
Fax (310) 450-1010 



"Clockwork" [motion picture] 
Producer/Director, Eric Breibert 
1 film reel (25 min.), 16mm, 1982 
Approximately $450 purchase, $50 rental 



Distributed by 

California Newsreel 
149 Ninth Street, Suite 420 
San Francisco, CA 94103 
Phone (415)621-6196 
Fax (415) 621-6522 



Summary: Through archival and contemporary documentary footage, studies the role of Frederick 
Winslow Taylor in the development of scientific management in the early twentieth century. 
Explains the techniques Taylor used to study worker movements and the methods he conceived to 
divide complex jobs into smaller, unskilled routines. 
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LEARNING OBJECTIVE WORKSHEET 



PROGRAM/COURSE TITLE: 


TASK NO: 
N/A 


UNIT/BLOCK OF INSTRUCTION: 
Individual project 


FILL 

OUT 

ONE 

SECTION 

ONLY 


□ 

TERMINAL 


TERMINAL OBJECTIVE NO: 


ENABLING OBJECTIVE NUMBERS SUPPORTIVE: 


□ 

ENABLING 


ENABLING OBJECTIVE NO: 


TERMINAL OBJECTIVE NUMBER SUPPORTED: 



BEHAVIORAL ACTION STATEMENT: 
Prepare duplicated documents 



PERFORMANCE CONDITION(S): 

1 . From books, journals, magazines, etc. 

2. Using a photocopier 

3. Using a stapler 

4. Using a yellow highlighter 

5. Using a PC or typewriter 

6. Given a copy of assignment sheet number 1, containing a list of subjects/topics 

7. Using a personal copy of the APA Publication Manual, 4 th edition 



ATTAINMENT STANDARD(S): 

1 . Submit a minimum of four (4) different duplicated documents 

2. Submitted at regular intervals during first 7 class sessions 

3. Carefully selected for quality & relevance to a particular subject/topic taken from list on assignment sheet no. 1 

4. Points important to subject/topic identified are highlighted 

5. Submittals to include on a cover page: (a) source reference in accordance with APA, 4 th edition; (b) name, SSN — last 
four; & (c) appropriate subject/topic, & its number 

6. Each submittal to be complete 

7. Tidy work 

8. Free from errors in spelling, grammar, & punctuation 

9. Prepared with assistance as necessary 

10. Submitted on 



TEST ITEMS: 



BEST COPY AVAILABLE 
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LEARNING OBJECTIVE WORKSHEET 



PROGRAM/COURSE TITLE: 


TASK NO: 
N/A 


UNIT/BLOCK OF INSTRUCTION: 


FILL 

OUT 

ONE 

SECTION 

ONLY 


□ 

TERMINAL 


TERMINAL OBJECTIVE NO: 


ENABLING OBJECTIVE NUMBERS SUPPORTIVE: 


□ 

ENABLING 


ENABLING OBJECTIVE NO: 


TERMINAL OBJECTIVE NUMBER SUPPORTED: 



BEHAVIORAL ACTION STATEMENT: 
Upgrade course glossary 



PERFORMANCE CONDITION(S): 

1 . Using a personal copy of the APA Publication Manual, 4 th Edition 

2. Given course glossary and glossary of terms in textbook 

3 . Using relevant sources — books, journals, magazines, etc. 

4. Using a PC or typewriter 

5. Using a stapler 



ATTAINMENT STANDARD(S): 

1 . Make beneficial changes to entries on course glossary 

2. Add new words and terms relevant to course 

3. Credit sources of new words and terms as well as beneficial changes, within text, in accordance with APA paragraphs 
3.34 - 3.40 (pp. 95-98) and 3.101 -3.102 (pp. 173-174) 

4. All references cited in text appear in the reference list in accordance with APA paragraphs 3.104 - 3.1 17 and Appendix 
3-A, pp. 189-222 

5. Submit neat materials, free from errors in spelling, grammar, & punctuation 

6. Submit with cover page including name, last four social security numbers, & course name & number 

7. Submit on 



TEST ITEMS: 
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Glossary 

The clear definition of terminology enhances communication. Therefore, the following 
keywords, terms, and phrases are defined for use throughout this document. 

ABILITY 

Power to do something; the result of learning and practice. A characteristic indicative of 
an individual’s skill, expertness, or competence. 

ACCOUNTABILITY 

Responsibility for the commitment of resources in terms of program results or outcomes. 
This accounting involves both the stewardship of resources and the evaluation of 
achievements in relation to specified objectives. 

ACHIEVEMENT 

What a person knows or can do after training. 

ACHIEVEMENT TEST 

An instrument that measures the current status of an individual’s proficiency in given 
knowledge or skills. 

ALIGNMENT 

The linking (aligning) of performance development and measurement to organizational 
goals and objectives. 
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APPLIED RESEARCH 

Research conducted for the purpose of applying or testing theory and evaluating its 
usefulness in solving problems. 

APTITUDE 

The natural ability or inclination to acquire knowledge and skills or to show potential for 
success in skills or an occupation when given training. 

ASSESSMENT 

The act of determining the significance, value, or amount of learning. 

ASSUMPTION 

Important fact(s) presumed to be true but not actually verified; assumptions should be 
described in the procedures section of a research plan and report. 

ATTITUDE 

A feeling or emotion that influences an individual’s responses. A disposition or tendency 
to respond positively or negatively toward a specific activity, event, idea, object, person, 
situation, thing, etc. 

BARRIERS TO TRANSFER 

Actual and perceived factors that act as impediments to transfer of training to the 
workplace. 

BASELINE 

Valid and reliable information and data about the intended trainee population, used to 
ascertain differences between performance before and after instruction. 
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BEHAVIOR 

An individual’s [visible, audible] action, performance, or product which can be measured 
by an observer according to specific and discrete criteria. 

BENCHMARKING 

A systematic approach for measuring and comparing the processes and products of one 
organization to those of another, which is considered to have the best practices 
(benchmarks), for the purpose of identifying practices in the first organization that can be 
improved. 

BIAS 

A mental leaning, partiality, or inclination. Bias in measurement occurs when 
characteristics of the measures, the process, or interpretation of the results leads to 
inaccurate inferences about the knowledge, skills, or other attributes of an individual or a 
group. Sample bias results from the omission or the unequal selection of members of the 
population without appropriate weighting. 

CENSUS SURVEY 

Descriptive study that attempts to acquire data from every member of a population. 

COMPARATIVE EVALUATIONS 

Evaluations that assess the effectiveness of a program by comparing it with an alternate 
program designed and developed to accomplish the same results. 
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COMPETENCY 

A cluster of related knowledge, skills, and attitudes that affect one’s job, that correlates 
with performance on the job, that can be measured against accepted standards, and that 
can be improved through training and development. 

CONTENT VALIDITY 

An assessment that attests that a product will produce the desired results. A comparison 
of learning objectives, instructional materials, and criterion tests to ensure that they track 
with one another and may be expected to produce the desired outcomes. 

CONTROL GROUP 

A group, similar on as many variables as possible, to the one that participated in the 
training being evaluated, but which was treated as usual (did not receive the training). 

CORRELATIONAL RESEARCH 

Collection and analysis of data in order to determine whether, and to what degree, a 
relationship exists between two or more quantifiable variables. 

COST-BENEFIT ANALYSIS 

An assessment of the specific costs and benefits (expressed in economic terms) involved 
with training. An analytic approach to making decisions. Requires the definition of 
objectives and identification of alternative way(s) of reaching each objective. A 
comparison can then be made by looking at the identification, for every objective, of the 
alternative(s) and seeing which results in the greatest benefit for a given cost or produces 
the expected level of benefits at the lowest cost. 




5 



COST EFFECTIVENESS 

A comparative evaluation derived from an analysis of two or more alternatives (actions, 
methods, approaches, equipment, support systems, etc.) in terms of the interrelated 
influences of cost and effectiveness in accomplishing a learning objective. 

COVERT BEHAVIOR 

Mental activity that is not directly observable; usually referred to as thinking. 

CRITERION (CRITERIA) 

The standard, rule, or test against which things, including learning, are measured and/or 
judged. 

CRITERION-REFERENCED PERFORMANCE TEST (CRPT) 

A sample work situation in which individuals perform a task which requires them to 
demonstrate that they have acquired the necessary knowledge and skills. Individual 
performance is compared to set attainment standards derived from a task analysis. Also 
called a Job Performance Measure. 

CRITERION-RELATED VALIDITY 

Validity determined by relating performance on a test to performance on another 
criterion. 

DATA 

Numerical values, gathered by using instruments (e.g., a questionnaire), from which 
conclusions can be drawn. 

DEFICIENCY 

Failure of an individual or group to meet a standard set of requirements. 

O 

ERIC 



23 



23 



6 



DEPENDENT VARIABLE 

The change or difference in behavior that occurs as a result of an independent variable; 
also referred to as the criterion variable, the effect, outcome, or post-test. 

EFFECTIVENESS 

Evaluation criteria based on how well training accomplishes what was intended — 
attainment of learning objective(s). Extent to which training outcomes match training or 
organizational goals. How well someone or something is performing compared to 
expectations. 

EFFICIENCY 

Resource control; doing more or the same for less money, time, etc.; reducing or 
containing costs. Also, reducing the inefficiencies of learning. Evaluation at the 
efficiency level is process-oriented and formative in nature. 

ENTERPRISE 

A business, company, commercial or industrial establishment, firm, or organization that 
employs workers. 

ENTRY TEST 

A test on learning objectives that the intended trainee must have attained in order to begin 
the training. 

EVALUATION 

The process of collecting information/data using tests, CRPTs, etc. and interpreting it in 
order to make decisions/judgments (as to whether training produced the desired 
effect/results). 
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EVALUATION INSTRUMENTS 

Tests, rating forms, checklists, reaction forms/opinionnaires, questionnaires, interview 
guides/schedules, and other devices used to (a) gather information and data; (b) determine 
achievement; or (c) determine the relative standing of an individual, group, or objective 
(attitude, performance, etc.). 

EVALUATION REPORT 

A written report documenting the purposes, objectives, procedures, conclusions, and 
recommendations from a course or program evaluation. Used by stakeholders to improve 
training and learning. 

EVALUATION SCHEMA 

A carefully devised plan (in outline form) for evaluating training courses and programs. It 
shows the different parts and the actions involved. 

EVIDENCE 

The result of design, measurement, and analysis, and the reporting of data pertaining to 
features of the program and its effects. 

EXTERNAL EVALUATION 

The collection and analysis of feedback information/data from outside the instructional 
setting to (a) evaluate the graduates’ job performance, and (b) determine whether the job 
requirements have changed. 

EXTRANEOUS INFLUENCES 

Factors affecting variables being measured, but which are not accounted for in the 
evaluation and which may have escaped the evaluator’s awareness. 
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FEASIBILITY STUDY 

An analysis conducted to establish the practicality and cost justification of a training 
course/program. 

FEEDBACK 

Information on performance is “fed” back to the (a) individual, in order to improve 
proficiency; (b) instructional designer, so that the materials and procedures can be 
improved on the basis of trainee needs; and (c) management system, so it can monitor the 
internal and external integrity of the instruction and make appropriate revisions. 

CENSUS SURVEY 

Descriptive study that attempts to acquire data from every member of a population. 
FORMATIVE ASSESSMENT/EVALUATION 

The process of collecting information and data on instructional materials, media, 
procedures, instruction, and tests, while they are being developed and tried out, in order 
to improve their relevance, effectiveness, and efficiency. The process begins during job 
analysis and continues through developmental testing. 

FOLLOW-UP EVALUATION 

Studies that examine the impact of training on job performance and the organization. 

GAIN SCORES 

The numerical difference between an individual’s post-test and pre-test scores. The 
results are an estimate of training effectiveness (learning). 
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GO/NO-GO 

Evaluation criterion whereby an individual is either 100% correct (go) or incorrect (no- 
go). Also called pass/fail. 

GRADING 

The process of assigning a mark or rating to trainee deliverables and examinations. 

HALO EFFECT 

The phenomenon whereby impressions concerning an individual (positive or negative) 
affect subsequent measurements of that individual’s work. 

IMPACT 

Effects of learning on trainees’ behavior and attitudes. 

INSTRUCTIONAL MATERIALS 

Tangible trainee-oriented resources, with instructional content. They facilitate the 
learning process and are an essential component of effective and efficient training courses 
and programs. Instructional materials can be categorized as (a) printed materials, 

(b) audiovisual media, and (c) manipulative aids. 

INSTRUCTIONAL METHODS 

Methods (strategies) of presentation, practice, and evaluation specifying the ways in 
which the desired learning outcomes are to be achieved in the instructional setting. 
Appropriate methods include lecture/discussion, demonstration, practical application, 
group or panel discussions, role playing, case studies, programmed instruction, and so 
forth. 
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INSTRUCTIONAL SETTING 

The place/environment in which instruction is provided. For example, the 

(a) schoolhouse — classroom, shop, laboratory, library, etc.; and (b) workplace — 

factory, construction site, garage, supermarket, hospital, etc. 

INSTRUCTOR 

A person who instructs; teacher or trainer. One who directs the development of 
individuals by helping them gain proficiency in skills through instruction, coaching, or 
facilitation to accomplish learning objectives. 

INSTRUMENTATION 

Selecting or developing measuring devices and methods appropriate to a given evaluation 
problem. 

INSTRUMENTATION ERROR 

A built-in bias that causes an instrument to be inaccurate. 

INSTRUMENTS 

Questionnaires, interview guides/schedules, and opinionnaires or reaction forms used to 
gather information and data from respondents. 

INTERNAL EVALUATION 

The collection and analysis of feedback and management information/data from within 
the instructional setting to assess training effectiveness in terms of individual attainment 
of learning objectives. 

ITEM 

One question or statement on an instrument. 
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JOB 

All the duties and tasks performed by a single job incumbent constitute that individual’s 
job. If identical tasks are performed by several individuals, they all hold the same job. 

The job is the basic unit used in carrying out the personnel actions of recruitment, 
selection, training, classification, and assignment. 

JOB ANALYSIS 

The process used to identify the tasks and elements necessary to perform a clearly- 
defined, specific job. Often involves observations of workers and interviews with those 
who know the job, in order to describe completely and accurately the work performed and 
the requirements for successful performance. 

KNOWLEDGE 

Techniques, principles, facts, information, etc., that, although not directly observable, are 
required to develop the skills and attitudes to effectively accomplish prescribed tasks and 
jobs. Knowledge is acquired by investigation, observation, and interaction with the 
environment (experiences). 

LEARNING 

Changes in the behavior of an individual, as a result of study and experience, that are 
retained and used. The changes in behavior are referred to as learning outcomes — 
acquisition of knowledge, skills, or attitudes. 

LEARNING OBJECTIVE 

A statement that specifies measurable behavior (performance) that trainees will be 
required to exhibit after instruction. Properly prepared, it includes the (a) behavioral 
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action (what the trainee does), (b) performance conditions (what the trainee is given), and 
(c) attainment standards (how well the trainee must perform the behavioral action). Also 
called a Performance, Behavioral, or Criterion Objective. 

LEARNING TAXONOMY 

Bloom’s Hierarchy — the taxonomic classification of cognitive, affective, and 
psychomotor learning behaviors/domains. 

LEVEL OF STATISTICAL SIGNIFICANCE 

The probability that the research results have occurred by chance. Example: p= .05 
means that the results will happen by chance only five times in a hundred (on the 
average). 

LIKERT SCALE 

Used to generate numerical values for qualitative questionnaire items, by using an attitude 
scale on which individuals rate the intensity of their agreement or disagreement with 
descriptors (e.g., strongly agree, agree, undecided, disagree, or strongly disagree). 

LIMITATIONS 

An aspect of a study which the researcher knows may negatively affect the results or 
generalizability of the results, but over which the researcher has no control. 
NORM-REFERENCED MEASUREMENT 

Determining (comparing) an individual’s achievement in relation to others’. 

OCCUPATION 

A skilled worker’s trade, craft, vocation, profession, or job. 
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OUTPUTS 

Influences training had on the trainee; products of the training process. 

OVERT BEHAVIOR 

Observable and measurable behavior/performance. 

PERFORMANCE AUDIT 

A framework for examining human performance problems and a set of procedures for 
determining the worth of correcting the problem, finding the cause, and designing cost- 
effective solutions to the problem. 

PERFORMANCE DEFICIENCY 

Inability to perform job tasks to established standards. 

PERFORMANCE EVALUATION 

The process of collecting information and data, by observation, interview, etc., and 
analyzing it to determine the level of proficiency or success in task performance. 

POST-TEST 

A test administered after the completion of instruction to assess whether an individual 
attained the learning objective(s). Also called a Post-Check or Post-Assessment. 

PREDICTIVE VALIDITY 

The degree to which a test is able to predict how well an individual will do in a future 
situation. 

PRE-TEST 

A test administered prior to instruction to determine what the individual already knows 
and can do. It is used to (a) confirm individual qualifications for entering a program, 
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(b) identify remedial training requirements, and (c) identify enabling 
objectives/instruction the trainee can bypass. The results are saved for comparison with a 
post-test in order to determine the amount of learning which occurred. Also called a Pre- 
Check or Pre-Assessment. 

PROFICIENCY 

The ability to perform a specific behavior (task or learning objective) to the established 
standard in order to demonstrate mastery of the behavior. 

PROGRESS TESTS 

Examinations administered periodically (daily, weekly, etc.) throughout the training 
course or program to evaluate trainee progress. 

QUALITY CONTROL 

Process of measuring and evaluating in order to maintain course or program standards 
through adjustments in instructional materials, methods, or setting. 

QUANTITATIVE 

Relating to or expressible in terms of quantity, or involving the measurement of quantity 
or amount. 

RANDOM SAMPLE 

A portion of the target population in which there is a high degree of probability that it 
contains at least some target population characteristics. Sample selection is based on 
proven random sampling techniques. 

RATING 

A judgment about a trainee’s knowledge, skills, or attitude. 
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REACTIONS 

Opinions, feelings, and perspectives of trainees regarding an instructional experience. 

RELIABILITY 

The extent to which (a) a test measures achievement accurately on successive trials with 
similar trainees, and (b) an instrument produces the same results when used repeatedly by 
the same person or group. 

SAMPLE 

A number of individuals selected from a population for a study in such a way that they 
represent the larger group from which they were selected. 

SATISFACTION 

Degree to which an individual is pleased with a learning experience, e.g., whether 
personal needs were met. 

SELF-EFFICACY 

An individual’s judgments of their capabilities to execute courses of action required to 
attain learning objectives and perform tasks on the job after training. It is concerned with 
judgments of what one can do with the skills possessed. 

SELF-REPORT 

Judgments made by trainees regarding their knowledge, skills, and attitude. 

SIGNIFICANT 

A real difference attributable to differences or relationships between two or more groups 
or individuals and not a difference due to chance alone. 
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SKILLS 

Involve physical or manipulative activities. All skills are actions having special 
requirements for speed, accuracy, or coordination. They typically require knowledge, 
training, and practice for their proficient execution. 

STAKEHOLDER 

An individual who (a) has an interest in the course or program being evaluated, (b) needs 
the results, or (c) is affected by the process or recommendations. 

STANDARDS 

Describes the measurable criteria or standards of performance which must be attained. 
Included in learning objectives. 

STRUCTURED INTERVIEW 

A carefully planned interview using written questions and instructions specifying how to 
conduct the interview. 

SUMMATIVE EVALUATION 

An evaluation performed after training is completed in order to determine the “summed” 
(overall) effectiveness of a course or program. 

SURVEY 

An attempt to collect data from members of a population in order to determine the current 
status of that population with respect to one or more variables. 

TASK 

A distinct measurable work activity that constitutes specific and necessary action by a job 
incumbent for a meaningful purpose. It is the work unit that deals with the methods, 
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procedures, and techniques by which parts of a job are carried out. Tasks vary in 
complexity but are made up of at least two elements. Related tasks are grouped together 
to make up a duty. 

TEST 

A device or technique used to measure an individual’s knowledge, skill, or performance. 

TRAINING 

Preparing individuals with the practical know-how and theoretical insights, through 
applied learning experiences in a schoolhouse or at a workplace, necessary for gainful 
employment as skilled workers, technicians, or entry-level professionals. It imparts the 
knowledge, skills, attitudes, understandings, work habits, and appreciations needed to 
enter and progress in an occupation. 

TRANSFER 

The effective and continuing application, on the job or in other settings, of the 
knowledge, skills, and attitudes acquired in training. Transfer is determined by a follow- 
up evaluation after the completion of training. 

VALIDATION 

The process of developmental testing and revision of instruction until it is effective in 
realizing its intent. 

VALIDITY 

The extent to which (a) a test measures what it claims to measure, and (b) an instrument 
gathers the information and data it is designed (intended) to gather. Types of instrument 
validity include 
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• Content — the extent to which the test represents the program content (face validity) 

• Construct — the extent to which the test represents the construct it purports to measure 
(attitude, skill, ability) 

• Concurrent — the extent of agreement with another test 

• Predictive — the extent to which future behavior can be predicted 



A function of the contribution that training makes to the organization. Determined by the 
extent to which there is a need for the contribution. 

WORKPLACE 

The enterprise in which graduates are employed to perform the skills trained. 

WRITTEN TEST 

A test in which individuals demonstrate their capabilities by written techniques. Not 
usually a performance test and, hence, generally a measure of supporting knowledge 
rather than skills. 



Note. Some entries in this Glossary were adapted from Training Course/Program Evaluation: 
Principles and Practices (pp. 334-336), by C. P. Campbell, 1998, Bradford, West Yorkshire, 
England: MCB University Press. 
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LEARNING OBJECTIVE WORKSHEET 



PROGRAM/COURSE TITLE: 


TASK NO: 
N/A 


UNIT/BLOCK OF INSTRUCTION: 
Individual project 


FILL 

OUT 

ONE 

SECTION 

ONLY 


□ 

TERMINAL 


TERMINAL OBJECTIVE NO: 


ENABLING OBJECTIVE NUMBERS SUPPORTIVE: 


□ 

ENABLING 


ENABLING OBJECTIVE NO: 


TERMINAL OBJECTIVE NUMBER SUPPORTED: 



BEHAVIORAL ACTION STATEMENT: 
Write a scholarly paper 



PERFORMANCE CONDITION(S): 

1 . Using a personal copy of the APA Publication Manual, 4 th Edition 

2. Given a copy of Assignment Sheet No. 1 

3. Using a PC or typewriter 

4. Using a stapler 



ATTAINMENT STANDARD(S): 

1 . Subject/topic of paper from list on Assignment Sheet No. 1 

2. Scholarship format & style in conformance with criteria specified in APA Publication Manual, 4 lh Edition, including 
chapter 4 

3. Used headings as an outline to reveal organization of paper 

4. Free from errors in spelling, grammar, & punctuation 

5. Used plain language; clear, simple, easily understood words & phrases 

6. Careful selection & consistent use of appropriate words & terms 

7. Cited sources within the text in accordance with APA, paragraphs 3.34 - 3.39 and 3.101 - 3.102 

8. Included at least one table or figure, also referred to in text 

9. Included a list of references (several) 

10. No less than 6 & no more than 10 pages of text 

1 1 . Included a glossary of key words & terms used in the paper 

12. No need to include (a) Author Notes, (b) Footnotes, or (c) an Abstract 

13. With assistance as required 

14. Submitted on 



TEST ITEMS: 
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Human Resource Development Department 501 The University of Tennessee 

Survey of Human Resource Development Information Sheet 



TIPS FOR SUCCESSFUL WRITING 



To meet requirements for effective written communications, study Chapter 2, Expression of 
Ideas, and Chapter 3, APA Editorial Style, in the Publication Manual of the American 
Psychological Association (4 th edition). In addition, consider the following guidelines: 

1 . Adequate time must be allowed for (a) planning, (b) information gathering, (c) writing, 

(d) review, and (e) revision, so as to meet the deadline without undue haste. 

2. In order to achieve effective and efficient communication, (a) write from an outline, 

(b) self-review the manuscript a day or so after it is written, and (c) have someone with the 
requisite skills critique the manuscript. Improvement is always possible. 

3. The subject, topic, or problem to be pursued should be stated in a clear and concise 
manner. 

4. Lead with the most important information — Don't bury the main point in the middle of a 
paragraph or the reader might miss it. To discover what your main point or main sentence is, read 
a paragraph. Now decide which sentence would be the last one you would toss out. That sentence 
is your main point for the paragraph. 

5. Make writing easy and pleasant to read by using the following by using the following 
techniques: (a) indent main points in a paragraph; (b) use boldface or italic type to emphasize 
important statements; (c) use lists and bullet points to create white space which rests the reader's 
eyes; and (d) use subheadings to organize the presentation and break space. Think of things you've 
read that appeal to you. Save those examples and use the devices you like in them. 

6. If you want your writing to be understood, keep the reader's needs, concerns, and point of 
view in mind as you write and revise. Avoid unfamiliar language or terms, and address questions 
you know the reader will logically have about your subject. 

7. Don’t overwrite and don’t be reluctant to cut. Every written communication should serve a 
useful purpose. Avoid overburdening the reader with unnecessary words and information. 

Consider the impact of the Declaration of Independence, the Gettysburg Address, and the Lord's 
Prayer. Remember, the reader's time is valuable. Never use a paragraph when a sentence will 

do — always strive for brevity. 

Be concise. Do not over-elaborate and extend the text beyond its effectiveness. Make your 
written communications as short as possible, yet maintain the clarity necessary to impart the 
necessary information. Two things happen if you are not concise — the reader may (a) not get to 
the crux of the problem because it seems a waste of time to wade through excess wordage, and 
(b) lose the emphasis that should be placed on various areas. 
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8. Use (a) one-, two-, or three-syllable words; (b) under twenty-word sentences; and (c) under 
ten-line paragraphs,. Anything more looks "too involved!" 

9. Avoid injecting your opinion. Reference the work of recognized experts on the subject in 
order to make your case. 

10. Quote directly what cannot be better said in your own words. Don’t forget to cite the 
source of quotes, information, illustrations, etc; consult the APA Manual. 

11. Keep first person "I" out of writing. Third person writing is powerful. 

12. Avoid qualifiers and vague modifiers — Don't use modifiers that almost say what you want 
when you can find ones that are exact. 

Close Exact 

The flight was very late. The flight was 4 hours late. 

The young trainee The 1 6-year-old trainee 

13. Don't use cliches (e.g., gut-level response, two-way street, cover all bases) — Worn-out 
expressions can wear a reader's patience thin. 

Trite Natural 

First and foremost First 

14. Avoid exaggeration — Avoid the superlative when you write. The greatest, the worst, the 
prettiest all leave you open to exception. To avoid arguments from the reader, avoid overstating 
your position. 

15. Use correct grammar, spelling, capitalization, and punctuation. Rewrite awkward phrases 
and use easy-to-understand words with specific meaning. Typing and copy quality should be given 
careful attention. 

16. Sum it up — The last paragraph should summarize what you've been discussing. 



Use the methods and mechanisms of effective writing to the best of your ability. Bear in mind 
that self-evaluation and objectivity are absolutely necessary for you to grow in effectiveness. To 
become an accomplished writer takes practice. The more you write, the more you think about 
writing, and the more you read the good writing of others, the more successful you will be at 
writing. 
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HISTORY OF TRAINING 



Directions: Each of the following items contains four (4) alternatives, one of which provides the 

best answer. Circle the letter which precedes the best answer. 



1. A result of America's change from an agrarian 

to an industrial economy was 

A. the substitution of apprenticeship for most 
other types of training. 

B. a de-emphasis on agricultural training. 

C. reduced interest in training programs 
because of high unemployment. 

D. the establishment of vocational education in 
schools to augment apprentice training. 

2. History shows that the greatest stimulus to the 

growth of training in the United States has been 

A. wartime skilled workforce needs. 

B. the decreasing influence of the profit 
motive. 

C. the prosperity of the 1920's. 

D. skilled worker supply/demand imbalances. 

3. The program to train shipyard workers during 

World War I was based on the principle of 

A. classroom training separated from the 
worksite. 

B. traditional apprenticeship training. 

C. show, tell, do and check. 

D. avoiding the use of work supervisors as 
trainers. 

4. Which of the following is true of the Job 

Instructor Training program of World War II? 

A. It was oriented to first- and second-line 
supervisors. 

B. It taught how to instruct and emphasized 
human relations between the supervisor and 
worker. 

C. It was originally designed for use in 
defense industries. 

D. All of the above. 



NOTES 
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TRADITIONAL BELIEFS BASED ON WISDOM AND COMMON SENSE 



Questions about workforce education and training were once answered by turning to the accumulated 
wisdom and common sense portrayed in the following old sayings. These tried-and-true adages may 
have been the earliest guiding principles (statements of best practices). Each of the 31 maxims listed 
here continues to provide a sound basis for practical action. 



SAYING: 

ADAGE: 

PROVERB: 

MAXIM: 

AXIOM: 



An adage, proverb, or maxim; expression of wisdom or truth 
An old saying tested over the years until popularly accepted as a truth 
A piece of practical wisdom expressed in homely, concrete terms 

A concisely expressed principle drawn from practical experience and serving as a rule; a 
popularly accepted saying that contains the wisdom of the past and many nations. 

A statement accepted as true, because it is obvious/self-evident. 



1 . Learn by doing (We learn to do by doing) 

2. Learn from experience (Experience is the 
best teacher) 

3. Practice makes perfect 

4. Skill comes from drill 

5. Use it or lose it (If you don’t use it, you 
will lose it) 

6. We learn best what is meaningful 

7. A picture is worth a thousand words 

8. Seeing is believing 

9. Plan your work; work your plan 

10. A stitch in time saves nine 



17. Nothing motivates like success 

18. Being ignorant is not so much a shame as 
being unwilling to learn 

19. Knowledge is power 

20. School ends, but education doesn’t 

21. You are never too old to learn 

22. The first thing to learn in school is how to 
learn 

23. A wise man is a good listener 

24. Unlearning is harder than learning 

25. Before you get to the three R’s, you’ve got 
to master the three L’s — look, listen, and 
learn 



1 1 . Haste makes waste 



26. Learn from your mistakes 



1 2. Better late than never 



27. Two heads are better than one 



13. If it's worth doing, it's worth doing right 



28. You cannot teach what you don't know 



1 4. It’s easier to do it right the first time than 
to explain why you didn’t 



29. You cannot teach a man anything; you can 
only help him to find it within himself 



1 5. Never enough time to do it right the first 
time, but always time to do it over (If you 
don’t have time to do it right, when will 
you have time to do it over?) 



O 
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1 6. It’s better to aim for excellent and hit 

good than to aim for good and hit average 



30. You can lead a horse to water, but you 
can’t make him drink 

31. You can tell the quality of the workman 
by the quality of his tools 
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HIGHLIGHTS OF EARLY UNIONISM IN THE U.S.A. 



1. Workers reacted to the disadvantages of their roles by forming producers’ or 
consumers’ cooperatives designed to restrain management control, by political action 
aimed at (a) overthrowing the wage system, and (b) bringing pressure on employers 
by forming unions. 

2. The union movement in the U.S.A. began in the latter part of the 18th century with 
associations of workingmen. They pledged not to work alongside those who did not 
serve an apprenticeship. 

3. First genuine strike occurred in 1786 by printers in Philadelphia. 

4. Between 1792 and 1794, the first continuous workers’ organizations appeared. 
Shoemakers in Philadelphia are an example. 

5. Associations of masters fought unions through the courts, using the charge of 
conspiracy. Journeymen shoemakers were convicted of joining a conspiracy because 
they went on strike in 1806 and used collective bargaining to gain higher wages. 

6. After the War of 1812, there was a long period of dominance by mercantile 
capitalism. Mills appeared alongside shop and home production. 

7. Trade associations or unions were formed or reformed in many crafts. After 1820, 
strikes occurred in (a) craft shops and (b) mills (factories). 

8. Early union unrest was a response to: 

• Mercantile employers’ cost cutting by holding down wages, lengthening the 
work day, breaking down the master- worker relationship, and introducing 
women and children into industry. 

• Atmosphere of Jacksonian democracy, with its war against the aristocracy and 
exalting of common people. 

9. Under adverse circumstances, workers organized against bankers and other 
aristocrats. They demanded an end to social inequality, imprisonment for debt, and 
child labor in New England mills. 

10. Equality of opportunity was the workers’ slogan, and the key, they thought, was a 
system of free education. 
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11. In the early 1830s labor moved, for a time, toward politics: 

• Workingmen’s parties were formed in various cities. 

• Aim was to get labor representatives into the legislature. 

• Parties disappeared and labor merged with the Jacksonian movement. 

12. The American workforce had no aims as a class; rather, workers wished to better 
themselves within the framework of capitalism and share in its benefits. 

13. National Labor Union — formed in 1866 

• Composed of national, local, and city assemblies of trade unions and reformist 
movements. 

• Politically-minded federal employees campaigned for 8-hour day, which was 
granted in 1868. 

14. First national strike occurred in 1877 by railroad workers. Touched off by a 
reduction in wages by the Baltimore and Ohio Railroad. 

• Militia called out to keep trains running. 

• Over 100 killed and several hundred wounded. 

15. Knights of Labor — founded by Uriah S. Stephens in 1869 

• Initially a secret organization of skilled workers. 

• Reformist organization, rather than a revolutionary one. 

• Demanded government ownership of railways and telegraph systems and 
graduated income tax. 

• Interested in producers’ cooperatives. 

• Favored boycott over strike. 

• Reached its peak in 1886. By 1893, only 75,000 of 700,000 left; virtually 
extinct by 1900. Causes: 

a. Inept leadership. 

b. Strikes ended disastrously. 

c. Working class split among racial, national, and religious groups, with 
employers exploiting these hostilities. 

16. Haymarket riot in Chicago — 1886 

• Org anisa tions of Socialist Labor Party sought to convince working class of the 
necessity for violent destruction of capitalism. 

• During struggle for 8-hour day at McCormick factory, lockout turned into a 
strike. 

• Fight between police and strikers led to a mass protest meeting; during the 
meeting, a bomb was thrown, killing several policemen. 

• Wave of hysteria broke over the country. Radicals were rounded up by the 
police. Eight leaders were arrested, tried, and convicted. 

• Consequences: (a) enabled employers to open a counteroffensive against labor 
movements in general, (b) convinced labor leaders of necessity to stay clear of 
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radicalism, and (c) symbolized possibilities and limitations of radical action by 
workers and the determination and ability of conservative forces to resist 
radicalism. 

17. Breaking point between the Knights of Labor and the craft unions was reached during 
Haymarket riot. 

• Leadership of Knights of Labor opposed the membership’s desire for a strike. 

• During a conference in 1886, delegates decided to form a new organization, 
the American Federation of Labor (AFL). Samuel Gompers was elected first 
president. AFL interested primarily in higher wages, greater job security, and 
better working conditions. 

18. Craft union (AFL) — dominated labor movement until middle 1930s. 

• Samuel Gompers and Adolph Strasser built a strong union among cigar 
makers. 

• Gompers abandoned interest in revolutionary ideals and became an opponent of 
socialism. He typified conservative trade unionism in the U.S.A. 

19. AFL superior to Knights of Labor 

• AFL provided for union autonomy, based on craft principles; suited the non- 
class-conscious American worker. 

• Proved that they knew how to organize and win strikes by limiting their 
objectives and size. 

• Dissociated themselves from radicals. 

20. Homestead Strike — 1892 battle between the Association of Iron and Steel Workers 
(an AFL union) and the Carnegie Steel Company 

• Centered around plant in Homestead, Pennsylvania. 

• Decided to strike rather than accept terms of contract renewal. 

a. Company demanded that workers take a wage cut. 

b. Workers staged a strike when the company began a lockout. 

c. Company attempted to overcome the strike by bringing in Pinkerton 
detectives and more workers. A bloody battle resulted, killing 10. 

d. Strike leaders arrested on a murder charge. 

• Strike ended — November, 1892 — bitter defeat for labor. 

21. American Railway Union — formed in 1893 by railway workers who were 
dissatisfied with conservatism and ineffectiveness of the "brotherhoods" (former 
organization of railroad workers) 

• Members employed by Pullman Palace Car Company had numerous 
grievances. They resented company owning all houses, banks, and utilities. 

• Strike demands were for reinstatement of discharged members, lowering of 
rent on Pullman houses, and restoration of formerly higher wages. 
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• Court injunction declared picketing illegal, and strike leaders were jailed. The 
strike collapsed. 

22. Strike of United Mine Workers in 1902 

• Grievances — low pay, unfair methods of weighing coal, long hours, and 
dangerous working conditions. 

• Intervention by President T. Roosevelt resulted in strikers going back to work, 
pending the findings of an arbitration commission. 

• Consequences — several secondary concessions; main aim of recognition as a 
union was denied. 

23. Clayton Act amended anti-trust laws and penalties of injunction. AFL grew from 4.2 
million members in 1914 to 5.1 million by 1920. 

24. Post-WW I labor conditions 

• Steel strike began in September, 1919, weakened in January, 1920, then 
collapsed. 

• Depression of 1921 resulted in considerable decline of trade union 
membership. 

• Company unions formed and, by 1926, had about half as many members as 
AFL. 

• Workers in non-union plants subjected to "yellow-dog" contracts, binding them 
not to join a union. 

25. Depression years 

• Seven million people lost jobs by 1930; 15 million by 1932, and, thus, trade 
union membership declined more. 

• Norris-La Guardia Act passed in 1932, revoking power of judges to issue 
injunctions in labor disputes, and outlawing "yellow dog" contracts. 

• Wagner Act passed in 1935. National Labor Relations Board, an independent 
government agency, was formed to enforce provisions of Act. 

26. New Deal — Committee for Industrial Organization (CIO) 

• Formed by dissatisfied delegates of 1935 AFL convention. The controversy 
which split the AFL and resulted in the formation of the CIO was largely a 
clash over personality and method rather than fundamental principles. 

• CIO unions were expelled from AFL, with exception of the Ladies’ Garment 
Workers. 

• Rubber workers in Akron joined a union and went on strike, followed by 
General Motors workers in 1936/37 with a "sit-down" strike, and then by U. 
S. Steel Company workers. 

• CIO transformed itself into the Congress of Industrial Organizations in May, 
1938, with basic principles and structure much like AFL. 
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27. Labor during World War II 

• The War Labor Board was set up and labor yielded right to strike. 

• Smith-Connally Act (a) gave President the right to seize any plant involved in 
war production, (b) demanded a 30-day notice of strike, and (c) forbade labor 
to contribute money to political campaigns. 

28. Labor since World War II 

• Greatest outbreak of strikes in American history — 1946. 

• Taft-Hartley Act — 1947, restrictive legislation which cut union power. 

a. Prohibited unions from coercing members. 

b. Prohibited secondary boycott. 

c. Prohibited jurisdictional strikes. 

d. Banned closed shops. 

e. Granted employees the right to call plant elections. 

• AFL and CIO merged — 1955 

• Landrum-Griffin Act — 1959, provisions covered union elections, disciplining 
members, rights of members, trusteeships over local unions, etc. 

29. A Comment . Perhaps the greatest single contribution of collective bargaining has 
been to insure equitable treatment of workers by employers. In any event, unions 
have provided improved conditions to workers and wages that are higher than non- 
union workers receive. Nevertheless, membership has declined to about 15 % of the 
nation’s workforce. However, based on current evidence, it appears that workers of 
the future will continue to want some control over their destiny, especially when 
productivity and profits increase while working conditions and job security decline. 
Without unions, workers would probably suffer losses beyond the growing disparity 
between what they are paid and the benefits received by top management. 
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Human Resource Development Department 501 
Survey of Human Resource Development 



The University of Tennessee 
Progress test No. 2 



BEHAVIORAL SCIENCES IN TRAINING AND DEVELOPMENT 



Directions: Each of the following items contains four (4) alternatives, one of which provides the 

best answer. Circle the letter which precedes the best answer. 



1 . The Hawthorne studies from 1927-32 found 
that 

A. physical and environmental influences have 
little effect on worker efficiency. 

B. social interactions and group influences 
may have a stronger effect on work 
efficiency than physical factors in the 
environment. 

C. experimental methods are of little value in 
improving work effectiveness. 

D. higher wages are the best incentive for 
increased productivity. 

2. Who is regarded as the "father of scientific 
management"? 

A. Elton Mayo 

B. W. Edwards Deming 

C. Frederick Taylor 

D. Samuel Gompers 

3. One argument in favor of Douglas McGregor's 
Theory Y philosophy is that 

A. Theory Y allows a range of management 
approaches; Theory X permits only one. 

B. subordinates will always use self-direction 
in meeting organizational goals. 

C. employees will always seek responsibility. 

D . achieving security is the main motivation of 
employees. 

4. Maslow's hierarchy of human needs predicts 
that a worker whose physiological, safety, 
social, and ego needs were satisfied would be 
motivated by 

A. salary increases. 

B. chances to satisfy self-fulfillment needs. 

C. chances to satisfy belonging and acceptance 
needs. 

D. fear of punishment or loss of reward. 
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Human Resource Development Department 501 
Survey of Human Resource Development 



The University of Tennessee 
Information Sheet 



SCIENTIFIC MANAGEMENT 



Scientific management was introduced to the public in 1911 during hearings before the 
Interstate Commerce Commission concerning increases in railroad freight rates. Shippers opposed 
the rate increase, charging that, instead, the railroads should operate more efficiently. During the 
hearings, eleven successful engineers testified that through something called scientific management 
it was possible to reduce costs. As a result of media attention, the phrase “scientific management” 
and the name of its developer — Frederick Winslow Taylor — were soon commonly known. 

In 1878, at the age of 22, Taylor was hired as a laborer at the Midvale Steel Company in 
Pennsylvania. After completing an apprenticeship, he became a journeyman machinist. Following 
this, he was made gang boss in the lathe department, then foreman, master mechanic in charge of all 
maintenance, chief draftsman, and chief engineer — all within 6 years. During his 12 years at Midvale 
Steel, Taylor developed the system of shop management that was later to be called scientific 
management. 

“Soldiering” was an accepted practice at Midvale Steel and other plants at that time. There 
were two kinds of soldiering — natural and systematic. Natural soldiering is the result of the human 
instinct to take it easy; today we might call it loafing. Taylor felt that a good foreman could prevent 
this. However, systematic soldiering arose from the belief that there was only so much work to be 
done: if a worker exceeded accepted productivity norms, someone else would be deprived of a job. 
Solving this type of soldiering was a challenge. Taylor’s first approach was to establish piece rates, 
but this was not successful. 
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Taylor realized that the problem was not that the workforce didn’t want to work, but rather 
that they and management did not share the same view of what made up a fair day’s work for the 
wages paid. As a result, the workers and management argued. Consequently, Taylor set out to 
establish standards as to what constituted a fair day’s work. He conducted experiments to establish 
what his workers ought to be able to do with the equipment and materials they were provided. 

Taylor thought he could finish the experiment in 6 months; however, he was still working on the 
problem a quarter of a century later. 

To arrive at productivity standards, Taylor withdrew machine tools from the shop floor, 
appropriated scrap materials, and recruited a journeyman machinist to assist him. He observed and 
studied variations in the operation of the machine tools, such as belting, shafting, tooling, speeds, 
feeds, materials, and motions. His ultimate goal was to discover the combination of conditions that 
would maximize productivity. 

In addition to establishing productivity standards for Midvale Steel, Taylor’s studies led to 
publications such as his Notes on Belting . Among other by-products was the discovery that a stream 
of water, directed on the chip at the point where it was being cut from a workpiece, permitted an 
increase in cutting speed, thereby increasing production as much as 40%. These studies also led to 
the development of high-speed tool steel. 

In the years that followed, many industries began to utilize Taylor’s findings. Such concepts 
as unit-time study became the basis for planning and scheduling, better methods of inventory control, 
cost accounting, production control, and quality control. Some time later came micromotion study 
(study of repetitive operations by means of slow-motion pictures). 
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Taylor did not publish his management ideas until 1912 when the American Magazine hired 
him to write articles for technical audiences. An out-growth of one of his articles was a definition of 
scientific management that included the (a) scientific selection of workers, (b) scientific education 
and development of workers, and (c) friendly cooperation between management and the workforce. 
Also in 1912 Taylor appeared before a special committee of the House of Representatives to 
describe scientific management. This was his final statement of principles, as he died in 1915. 

One principle of scientific management is that the worker is to perform assigned tasks in an 
assigned manner. This is accomplished in the archetypal work organization with a high degree of 
specialization and division of labor. Each worker is given a fixed task to be repetitively performed. 
The worker develops expertise in performing that task, rather than the broader knowledge or more 
general skills that may accompany more varied tasks. In Taylor’s words, 

The most prominent single element in modern scientific management is the task idea. 

The work of every workman is fully planned out by the management ... and each man 
receives in most cases complete written instructions, describing in detail the task 
which he is to accomplish, as well as the means to be used in doing the work. This 
task specifies not only what is to be done, but how it is to be done and the exact time 
allowed for doing it. 

This work organization requires a hierarchical, regimented environment in which there are 
layers of management to assure that decisions are correctly executed. Through the “chain of 
command,” decisions are communicated to workers who are instructed in what they are expected to 
do. They are then closely monitored to assure that they do as they are told. 



C 2 99 



O 

ERIC 




52 



Peer Training & Teamwork 1 

PEER TRAINING AND TEAMWORK 
Clifton P. Campbell 
The University of Tennessee 

The premise is that, on the job, some of the most relevant and effective learning takes place 
efficiently worker to worker. For example, learning might best occur when an experienced worker 
shows a new employee how to use a tool correctly and safely, rather than waiting for a scheduled 
training session. This type of peer training happens naturally in the workplace and is, more often 
than not, spontaneous, need-specific, and therefore highly relevant. In addition, people rarely work 
in isolation: indeed, in today’s highly competitive global marketplace, teamwork is responsible for 
improving organizational performance. While “peer training” and “teamwork” are recognized 
methods of learning and working, they are not used in a combined form as often as they could be. 
Competitive Structure 

Formal education and training systems generally focus on studying and learning alone, 
rewarding with the highest grades those who are successful in their competition with classmates 
rather than those who cooperate with others and contribute to group success. It is often the case 
that an individual can attain a personal goal (an “A” grade, for example) only if others fail to attain 
theirs. 

For those who are competitive by nature, this traditional approach is acceptable. 
Nevertheless, it is common knowledge that when people cooperate on a project (without wasting 
energy and time on competitiveness), they can create powerful results, better than what most 
individuals, working on their own, could achieve. 

Initiating Cooperation 

At the beginning of a training program, some trainees may resist the idea of peer training 
and the support of team mates because of a reluctance to risk personal achievement by working 
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with others who may not share their goal. Comments from trainees regarding this reluctance 
include: 

“I don’t see the relevance of teams and prefer to work alone.” 

“Some will do little work, learn nothing, and yet receive the grade earned by team mates.” 

Some trainees are uncomfortable working with strangers. Admittedly, trainees with friends 
or acquaintances from the workplace or other training programs have an advantage at first. Once 
the “ice is broken,” however, many trainees have said they value learning and working with 
someone new. 

One activity that helps trainees get acquainted is having them prepare a notice (on a sheet of 
paper) that contains relevant information such as: 

1. What attributes they want in team mates (e.g., people who work hard and are 
motivated to achieve high grades, or possibly people that want to socialize) 

2. What they can offer a team (e.g., sense of humor; loyalty; previous experience with 
the instructor; computer, research, or editing skills) 

3. Topics/subjects in which they have expertise and interest 

4. Preferred method of communication — in-person meetings, telephone, faxes, e-mail, 
etc. 

5. The geographic locations where they live and work 

6. What days/times they could meet outside of class 

The notices can then be read or distributed to classmates, posted in the room, and so forth. When 
this activity takes place before trainees have even associated names with faces, it is advisable to 
have individuals include a recent photograph on the notice. 

Forming groups. Among the many ways to organize groups are 

1. Assign trainees to a group based on their ability, background, interest, personality, 
etc. (time-consuming, and requires the accumulation of information on which to base assignments) 
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2. Pull names out of a hat (this reduces trainee autonomy) 

3. List all trainees alphabetically, by their last name, then separate the list into a 
predetermined number of groups by (a) combining all A’s and B’s to form a group, all C’s and D’s 
to form another group, etc.; or (b) counting down the list 1, 2, 3, and then forming groups of all 
l’s, all 2’s, etc. 

4. Have trainees verbally count off (e. g., 1, 2, 3, 4, and then form teams of l’s, of 2’s, 
of 3’s, etc.) 

5. Combine trainees who live or work in the same geographic area (to facilitate in- 
person meetings) 

6. Combine trainees who prefer to use e-mail, fax, etc. rather than in-person meetings 

7. Invite trainees to team up with people they know 

8. Invite trainees to wander around the room “sensing” each other and chatting with 

people they “feel” they have empathy with 

9. Leave the method of choosing team mates (and the consequences) up to the class 

Group size. Two factors to be considered in determining group size are (a) the larger the 

group, the more personalities there are to deal with; and (b) there should be sufficient opportunities 
for each trainee to participate. Research on the size of decision-making groups found that as group 
size increased from 5 to 12, consensus resulting from discussion became more difficult and took 
longer. Furthermore, when groups grew larger than 12, communication and cohesion became 
problems which led to factionalism. When in doubt about group size — start small. Experience 
shows that groups of 3 to 5 work well. 

Uses of This Approach 

The peer training and teamwork approach may be used for a host of different activities: 

1 . Questioning one another’s generalizations, ideas, assumptions, understandings of 
what the instructor said, etc. 
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2. Aiding recall — ask each other the main points in the previous lecture/discussion, 
quiz one another on key concepts, and so forth 

3. Solving problems by drawing on one another’s skills, experiences, learning methods, 
and contacts 

4. Discussing assignments — verbally analyze the question(s) together, but write up 
the assignment separately 

5. Completing team assignments 

6. Task-related brainstorming 

7. Being alert for information useful to a team mate — exchange photocopies of 
material, for example 

8. “Showing the ropes” to newcomers 

9. Proofreading and critiquing each other’s assignments before they are submitted 

10. Offering constructive feedback and providing mutual support 

Moreover, researchers pointed out that when team members have complementary skills and 
expertise, their achievement is enhanced. In addition to this, Nancy Day wrote in “Informal 
Learning Gets Results” (in Workforce , June 1998), researchers found that teams were actually one 
of the richest sources of informal learning. 

The teamwork experience provides opportunities to develop and practice interpersonal and 
negotiation skills — two skills employers say they want. There are many ways in which team 
members have helped one another: 

“Working and learning within a group made me feel more secure because my team mates 

understood my concerns and problems.” 

“I appreciated the sense of identification with a group. It made me feel less lonely; I also 

enjoyed the responsibility of looking after someone else’s interests.” 
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Some unanticipated benefits were the way team mates strongly motivated and brought out the best 
in one another: 

“She gave me the push I needed and the motivation to keep going.” 

“He boosted my confidence when I was down.” 

Team Attributes 

The following are distinctive attributes of a cooperative and productive team. 

Agreement on high expectations. All members have a will to excel. Principal motivators 
are ethical behavior, high performance standards, quality, accuracy, and excellence. Mediocrity is 
not tolerated. The catchphrase is, “Until something meets our standards, we won’t submit it.” 

Commitment to common goals and a climate of trust. Goals provide team members with a 
common focus. When they are agreed to by a consensus (not a majority vote), all members accept 
them. Further, they realize that the goals can be achieved only through a collaborative team effort. 
In addition, all team members have an unquestioning belief in their team mates. Trust is the glue 
that holds the group together. 

Open communications and access to information. Team members realize that 
communication is important and necessary. They honestly and openly express their thoughts and 
feel free to ask questions relevant to the work being done. There are no hidden agendas; everything 
is above-board. Information is viewed as a vital resource, and the leader is responsible for assuring 
that all members have the information needed to accomplish their tasking. 

Assumed responsibility for work. The work is divided equitably, with each member having 
clearly defined tasks to perform. In addition, there is a commitment to do anything that needs 
doing. All members of the team hold themselves mutually accountable. 

Table 1 was prepared based on a review of the literature. It shows the outcomes that 
various authors have identified for the competitive and cooperative approaches to learning and 
working. 
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Table 1. Outcomes Attributed to Competitive and Cooperative Approaches 



Outcomes 



Competitive Cooperative 



A drive to excel in personal performance 


✓ 




Ownership, a need to be one’s own boss 


✓ 




Self-reliance 


✓ 




Efficient completion of learning and practice 
activities/projects 


✓ 




Higher retention plus transfer of information and skills to 
the workplace 




✓ 


Integration of analytical, innovative, and executive skills 




✓ 


Verbal abilities 




✓ 


Cross-functional: more than one way, out-of-the-box 
thinking 




✓ 


Creative ability: divergent and risk-taking, productive 




✓ 


controversy 

Interpersonal skills, leadership, communication, problem- 
solving, critical thinking, conflict resolution, etc. 




✓ 


Acceptance and appreciation of individual differences 




✓ 


Positive interdependence 




✓ 


A sense of belonging, feeling psychological acceptance 




✓ 


Sociability 




✓ 


Support, motivation 




✓ 


Personal accountability, recognition 


✓ 


✓ 


Mastery of concepts, principles, and processes 


✓ 


✓ 



Note. The outcomes are presented in random order by category. 
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A Comment 

Peer training and teamwork are not arbitrary notions or fads. The idea of people learning 
and working together dates back to the beginning of civilization, when survival depended upon the 
acquisition of food-gathering and defensive skills, as well as the ability of the group to get along 
with each other. A difference between the historical roots of peer training and teamwork and the 
process that exists today is the deliberate effort to create and maintain a small group of people who 
learn and work well together and enjoy doing so, while producing high-quality results. 

By encouraging trainees to learn and work together, rather than to compete with each 
other, we are responding in a positive way to contemporary workplace practices. Moreover, 
trainees do not feel so alone as they progress through learning activities. 



O 

ERiC 



66 



59 



Page 1 of 2 



Effective and Ineffective Teams 





Effective 




Ineffective 




Climate/Culture 




• 


Honest and open communication 


• 


Guarded or self-serving 


• 


Concern for all individuals 




communication 


• 


Commitment to team performance 


• 


Authority-directed problem solving 


• 


Trusting and flexible 


• 


Emphasis on rules and a rigid 


• 


Motivating and energizing 




attitude 





Participation 




• 


Everyone gets assignments and 


• 


Dominated by strong member(s) 




completes them on time 


• 


Some ideas are ignored or 


• 


All members listen to one another 




overridden 


• 


All ideas are given a fair hearing 


• 


Hidden agendas in contrast to team 


• 


Both participation and goal 




agenda 




achievement are emphasized 


• 


Members express indifference or 


• 


Time and effort are directed to 




hostility to others 




developing strong interpersonal 
relationships and building individual 
problem-solving skills 


• 


Individual members avoid 
assignments or submit poor quality 
work after the due date 





Leadership 




• 


Shared responsibility 


• 


Delegated by position or influence 


• 


All members feel responsible for 


• 


Position determines influence 




contributing to team goals 


• 


Power is concentrated in authority 


• 


Different members, because of their 




positions 




knowledge or abilities, act as 
"resource expert" at different times; 
thus leadership roles change as the 
tasks and needs of team change 


• 


Obedience to authority is the norm 


• 


As leadership roles change, all team 
members are supportive 






• 


Confidence in each member's 
abilities 







ERIC 




60 



Page 2 of 2 



Conflict Management 



Controversy is viewed as positive 
and essential to the problem-solving 
process 

Disagreements may be frequent and 
candid, but are also relatively 
comfortable 

No personal attacks; criticism is 
constructive and even supportive in 
nature 



Disagreements may be suppressed 
or "resolved" by a majority vote, 
which leaves an unconvinced 
minority 

Criticism is embarrassing and 
tension-producing, often leads to 
accommodation or compromise 



Decision 



Reached by consensus 
All members are customarily in 
agreement; however, there is little 
tendency for those in opposition to 
simply "go along" 

Disagreements are used 
constructively and formal voting is 
held to a minimum 



Group discussion or involvement 
may be solicited, but strong 
member(s) makes final decision 
Those in opposition are expected to 
conform even though they remain 
unconvinced 
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Title of Training Program: Surveying 



Coordinator: C. P. Smith 



Activity 


Activity 


Person(s) 


Start & due 


Work 


number 


name 


responsible 


dates 


weeks 






Planning 






01. 


Set dates/schedule training 


C. P. Smith 


5/10 - 5/22 




02. 


Organize curriculum team 


C. P. Smith 


5/16-5/22 


2 




Before Training 






03. 


Analyze training policy 


C. P. Smith 


5/30-5/31 




04. 


Analyze training needs 


R. W. Jones 


6/01 - 6/8 




05. 


Analyze the job 


G. C. Robins 


6/13 - 6/24 




06. 


Analyze target population 


T. W. Powell 


6/13 -6/17 




07. 


Develop learning objectives 


R. W. Jones 


6/17-7/13 


10 


08. 


Develop tests 


R. W. Jones 


7/14 - 7/27 




09. 


Specify instructional delivery 


T. W. Powell & 


7/25 - 7/29 






system 


R. W. Jones 






10. 


Organize resources 


T. W. Powell 


8/1 - 8/5 






During Training 






11. 


Instruct trainees 


T. W. Powell 


8/29 - 10/28 


9 


12. 


Evaluate trainees and program 


R. W. Jones 








After Training 






13. 


Analyze evaluation data 


T. W. Powell 


11/1 - 11/3 




14. 


Write evaluation report 


T. W. Powell 


11/4 - 11/10 


4 


15. 


Examine report findings 


C. P. Smith 


11/11 - 11/12 




16. 


Revise training curriculum 


T. W. Powell 


11/15 - 11/26 





Total = 25 



Example of a Scheduling Work Sheet 
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PLANNING WORKSHEET (MILESTONE CHART) 
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Human Resource Development 



The University of Tennessee 



TEAM MEMBER RATING FORM 



Project title 

Team member rated Rater 



Ratings: 




1 — Unacceptable 


Performance did not meet requirements. Add a 
comment to support your opinion. 


2 — Needs Improvement 


Performance met some but not all requirements. List 
improvement(s) needed in comment space. 


3 — Acceptable 


Performance met all requirements. 


4 — Excels 


Performance excelled in all areas. 


5 — Outstanding 


Performance exceeded all requirements and is of highest 
caliber. Add a comment to support your opinion. 



Categories: 

MEETING DEADLINES: 

Dependable and reliable, met deadlines in performance of group project. Managed and 
budgeted time. Accommodated needs of the team. 

Rating: Comment: 

PROFESSIONALISM: 

Implemented professional qualities into all actions. Assumed responsibility for work that 
must be done. Was prepared for and participated in meetings. 

Rating: Comment: 

TEAMWORK: 

Helped team meet its objectives. Willing and capable of leading team in proper direction. 
Helped to keep team focused and on task and did his or her share of the work. 

Rating: Comment: 

INTERPERSONAL SKILLS: 

Demonstrated communication, problem solving, and negotiation skills. 

Rating: Comment: 

Suggestions: 
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Human Resource Development 



The University of Tennessee 



TEAM MEMBER EVALUATION FORM 



Assignment Sheet No. Project Title 

Team member evaluated Evaluator 

Directions : Circle the number that, in your opinion, most closely corresponds to the team 
member's performance. 

Ineffective Effective 

Performance 1 2 3 4 5 



1 . Attended project meetings 1 

2. Arrived on time and stayed for entire 1 

meeting 

3. Was prepared for and participated in 1 

meetings 

4. Listened to comments by others, 1 

provided knowledgeable insights, and 
accepted constructive criticism 

5. Did his or her share of the work 1 

6. Performed quality work in a timely 1 

manner 

7. Exhibited sound judgment and decision- 1 

making skills 

8. Exhibited ability and willingness to 1 

provide leadership 

9. Was dependable/reliable 1 

10. Exhibited organizational and problem 1 

solving skills 

1 1 . Conveyed a professional/cooperative 1 

attitude 

12. Worked well with others 1 



2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 



3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 



4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 



5 

5 

5 

5 

5 

5 

5 

5 

5 

5 

5 

5 



Comments: 



O 

ERIC 



c 2 



65 



73 



Human Resource Development 



The University of Tennessee 



TEAM MEMBER RANKING 



Introduction: 

One problem of having people work in groups is that some contribute more than others to 
the success of their group’s project. It is impossible for the instructor to know all of what 
happened during the preparation process. 

One way of collecting information about individual contributions is to have each group 
member rank all others. Ranking is not an expression of grades such as A, B, C, etc. 

Instructions: 

Please rank order the contributions of the individuals in your group by writing the name 
of one member of your group on each of the lines provided below. Ranking should be 
assigned as follows: 

1 — the person who contributed the most to the group project 

2 — the person who contributed the next-most to the group project 

3 (or last ranking number) — the person who contributed the least to the group project 

Do not rank yourself. Assign each rank to only one individual (no "ties"). You are 
encouraged to add comments (under the heading Comments) to support your rankings. Do 
not share your rankings with others in your group or the class. 

When you have completed this ranking sheet, sign it in the space provided, fold it, and 
give it to your instructor. 

Rank Name 

1 

2 

3 

Comments: 
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ORGANIZATION AND MANAGEMENT OF TRAINING 



Directions: Each of the following items contains four (4) alternatives, one of which provides the 

best answer. Circle the letter which precedes the best answer. 



1 . Managers can conclude that a formal training 

program will be useful when 

A. specific skills are required for successful job 
performance. 

B. individuals’ performance does not meet job 
standards. 

C. the organization is not meeting goals included 
in annual or long-term plans. 

D. problems exist that can be solved by giving 
personnel identifiable items of knowledge, 
skill, or attitude. 

2. When originally organizing training efforts, the 

first task is to 

A. recruit staff for the training department. 

B. study the organization to determine training 
needs. 

C. decide on the internal organization of the 
training department. 

D. survey the training that competitors use. 

3. To perform the evaluation function successfully, 

A. courses and programs must have performance 
objectives. 

B. a comprehensive evaluation must be conducted. 

C. operating plans must be as long term as 
possible. 

D. training activities and staff must be organized 
by function. 



4. The most important function of a training policy is 
to 



A. describe training programs to be carried out. 

B. enumerate the problems that led to the 
establishment of a training program. 

C. show how training will help achieve 
organizational (workplace) goals. 

D. convince top management that training is 
necessary. 
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WORKPLACE GOALS THAT JUSTIFY INVESTING PRIVATE RESOURCES IN TRAINING 



The most important result of performance-based/results-oriented training is a measurable 
improvement in an individual's contribution to workplace/organizational goals (strategic 
objectives, business results). Those goals listed here are used in determining the benefits of 
training and/or the return on a training investment. 



1 .00 Improved/Increased/Enhanced/ 
Higher/More/Better/Greater 

1.01 Increased quantity (a) rate of 
production, (b) amount 
produced, (c) output 

1 .02 Improved quality (a) degree 
of excellence, 

(b) conformance to 
requirements, (c) durability 
(long-life), (d) fitness for use, 

(e) absence of defects, 

(f) reliability 



1.11 More variety 

1.12 Improved products 

1.13 Improved service 

1.14 Enhanced employee 
versatility and proficiency 

1.15 Improved safety and health 

1.16 Improved morale, personal 
satisfaction, attitude, 
motivation, trust 



1 .03 Higher standards 

1 .04 Improved accuracy 

(a) precision, (b) exactness 

1 .05 Increased productivity 

1 .06 Enhanced performance 

1 .07 Increased effectiveness 

1.08 Improved efficiency 

1 .09 Higher utilization of 
machines and equipment 



1.17 Greater dependability 

1.18 Improved teamwork 

1.19 Higher integrity and ethics 

1 .20 Increased employee 
satisfaction, identification 
with the organization 

1.21 Improved public image, 
relations 

1.22 Enhanced customer service 
satisfaction, retention 



ERIC 



1.10 Better use of resources, 
(a) money, (b) time, 

(c) personnel 



1 .23 Increased competitiveness 

1.24 Increased sales 
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1.25 


More market-driven 


2.09 


Fewer customer complaints 


1.26 


Better cost containment 


2.10 


Fewer accidents, medical 
and workers compensation 


1.27 


Increased profitability, 
economic growth, market 




claims 




share 


2.11 


Less tension, conflict, 
litigation (fewer grievances) 


Reduced/Decreased/Lower/Fewer/ 






Less 




2.12 


Fewer lawsuits and fines 


2.01 


Fewer faults/defects and 


2.13 


Less lateness and 




rejects (less rework) 




absenteeism 


2.02 


Fewer errors/mistakes 


2.14 


Fewer requests for transfer 


2.03 


Reduced scrap/waste/ 


2.15 


Reduced employee turnover 




materials costs 




(increased retention) 


2.04 


Less discarded paperwork 


2.16 


Reduced expenditures 


2.05 


Less damage to tools and 


2.17 


Lower production/operating 




machines (less downtime) 




costs 


2.06 


Decreased amount/ffequency 

of supervision 


2.18 


Reduced lead time 






2.19 


Reduced cycle time 


2.07 


Decreased labor cost, 
overtime, workforce 


2.20 


Reduced time to market 


2.08 


Reduced warranty expenses 







(buy backs) 
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WORKFORCE TRAINING 



Linking Training to Organizational Goals 

Most organizations are not in the business of training employees except when doing so 
contributes directly to their goals. Organizational (workplace) goals, also known as strategic 
objectives or business goals/objectives/results, which justify training include: 

1. Better quality (meet or exceed requirements, fitness for use, durability, reliability) 

2. Improved standards/accuracy 

3. Increased productivity /quantity /output/performance/efficiency 

4. Increased worker versatility and proficiency 

5. Improved morale/ attitude/dependability 

6. Improved safety and health 

7. Better products and service 

8. Better utilization of machines and equipment 

9. Fewer defects and rejects (less rework) 

10. Reduced scrap/waste/material costs 

1 1 . Less damage to tools and machines (less downtime) 

12. Lower operational/unit costs and increased profitability 

13. Decreased amount/frequency of supervision 

14. Less worker absenteeism and turnover 

15. Decreased customer complaints and warranty costs 

16. Fewer accidents and medical claims 

17. Less conflict, litigation (fewer grievances) 

It can be argued that some of the above organizational goals are not legitimate reasons for training; 
but rare indeed is the professional who would argue that none of these outcomes ever accrues to a 
training effort. 

Alternatives to Training 

Training tends to be the universally suggested solution to performance problems. While it 
is the appropriate activity when (a) required by a government regulation (compliance training) 

(b) new skills are required, or (c) existing skills are deficient, training cannot correct all 
organizational problems. Consequently, the following alternative interventions should be 
considered before training is prescribed. 

Changes in management and supervision, e.g., replace adversarial relationships between 
management and the workforce with cooperation, and improve supervision, 
mentoring/coaching/counseling 

Employee empowerment (involvement, decision making, responsibility, etc.) 

Improved methods of worker selection 

Communication of job performance requirements and standards, along with regular 

feedback n 

$ 
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5. Use of job performance aids 

6. Put existing knowledge and skills to their most effective and efficient use 

7. Reassigning employees 

8. Changes in job requirements (job redesign, enlargement/enrichment or job rotation) 

9. Better working conditions/environment (ergonomics) 

10. Recognition and reward of high level performance (financial incentives as well as other 
forms of appreciation) 

1 1 . Better pay and benefits 

12. Providing high quality tools, equipment, materials, and supplies 

13. Introduction of new technology and automation (machines, processes, etc.) 

14. Improved morale, motivation, and workplace harmony 

15. Attention to interpersonal, group, and intergroup behaviors 

16. Job security 

17. Advancement (promotion) based on merit and seniority 

18. Tactful discipline 

Distinguishing Between Education. Training, and Development 

Table 1 on page 3 facilitates a comparison of education, training, and development in 
terms of six factors (i.e., definition, purpose, expected outcome, scope, time frame, and 
relationships). It is reasonable to conclude that a training system differs from a formal education 
system in that training focuses on: 

1 . Specific requirements based on a needs assessment and job analysis 

2. The prompt application of knowledge and skills acquired 

3. Practice on operating equipment with supervision by an instructor or knowledgeable 
worker 

4. Direct relationships to a particular job 
Formal education systems, on the other hand, focus on: 

1 . Future applications of learning 

2. Indirect relationships to a job or occupation 

3. Implied learning for use in unpredictable situations 

4. Social coherence, successful living within a culture 

Finally, development deals with: 

1 . Projected or forecast needs of a nation or organization 

2. Orientation toward the entire economy or sectors of the economy 

3. Organizational requirements 

While their focuses differ, training is effective to the extent that: 

1. Graduates of the formal education system have acquired the basic knowledge, skills, 
and attitudes necessary as a foundation for job training. 

2. Instruction is based on performance requirements for jobs with favorable employment 
prospects. 
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Table 1 

A Comparison of Education. Training, and Development 



Education 


Training 


Development 




Definition 




Learning experiences to develop 
the whole person — mind, 
character, personality; learning 
concepts and rules, etc. by 
formal schooling 


The use of practical experiences 
and theoretical insights to 
prepare individuals for gainful 
employment as skilled workers 
or technicians 


Process of growth by which a 
nation or organization expands 
the number or modernizes the 
nature of its activities 




Purpose 




To promote a just society; impart 
knowledge, skills, and values 
needed in a complex culture; 
critical thinking; and academic 
excellence 


To supply the knowledge, skills, 
attitudes, and qualifications 
necessary to enter and progress 
in a particular job 


To assure organizations of a 
skilled workforce, technology, 
and equipment to meet 
previously established national 
goals 




Expected outcome 




Individuals with social skills and 
a foundation for lifelong learning 


Individuals who can perform job 
tasks proficiently in order to 
meet predetermined workplace 
goals: increased productivity, 
higher earnings, etc. 


Competitive organizations which 
(a) engage in new processes 
and/or products; and (b) acquire 
the systems, materials, and 
workforce to remain operative 




Scope 




Deals with broadly applicable 
abilities, attitudes, and values 


Deals with specific tasks, 
identified through a job analysis, 
requiring certain knowledge and 
skills 


Involves complex sets of tasks 
and requirements of the entire 
economy or sectors of the 
economy 




Time frame 




Long-term effort; deals with 
possible future applications of 
learning 


Deals with current needs and 
prompt application; short-term 
orientation is normal 


Deals with projected or forecast 
needs on a long-term basis 




Relationships 




Administrative interfaces with 
politicians, policy makers, 
special interest groups, and 
teachers 


Consultative interfaces among 
managers, workforce planners, 
job analysts, and a host of other 
specialists 


Consultative and administrative 
interfaces by policy makers, 
workforce planners, and other 
groups as appropriate 
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Training cannot and should not be completely separated from formal education, though their goals 
and approaches differ. 

The Role and Purpose of Workforce Training 

Privately-funded training is appropriate when there is a deficiency in knowledge and skills. 
This can happen when (a) new workers are hired, (b) new equipment is purchased, (c) a worker is 
promoted or transferred, (d) new jobs are created, (e) jobs are to be performed in new ways, or 
(f) jobs are being performed in deficient ways by incumbent workers. Publicly-funded training is 
appropriate when one or more of the following situations exist: 

1 . Workers with specific skills are not available in sufficient numbers in the present 
workforce. 

2. Changes in technology, methods, or required behaviors have caused current skills to be 
obsolete. 

3. Economic restructuring requires workers to move from one occupation to another. 

4. Unemployment rates are high. 

5. Parts of a population are not able to obtain employment in order to support themselves. 
Commentary 

The only valid result of training activity is a measurable increase or improvement in an 
individual’s contribution to the goals of an organization. Thus, the trained behavior must not only 
be observable and measurable, it must also be transferred to the job. Training is a means toward 
an end. The end is an improvement in the organization’s ability to meet its goals. The basic 
purpose of training is to shape or reshape the behavioral pattern of an individual. The desired 
function of a trainer (or training system) is to provide and manage the experiences in which 
learning can take place. 



Note. None of the numbered lists on this Information Sheet were sequenced in a methodological 
order. Their arrangement merely reflects an effort to present items logically. 
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THE IMPACT OF A TRAINING EXPERIENCE 



Training should be considered in light of an individual's prior experience and the behaviors 
which must be developed in the present or proposed environment. When a need for training exists, 
those responsible for instruction must be aware that the trainee does not arrive at the training 
situation needing to learn everything. A part of the behaviors (knowledge, skills, and attitudes) 
necessary to do the job will have been previously learned. In fact, everything trainees learn during 
the training experience will form only a part of their total "experience bank," no matter how 
comprehensive the particular course or program is. This point is illustrated by the formula: M - I 
= D. In this equation, M signifies Mastery of all necessary behaviors, and I represents Inventory 
of existing behaviors. D then represents Deficiency or training need. 

The "Training Fraction" shown below provides an indicator of the impact any training 
experience will have on an individual's total experience. The fraction indicates that the training 
experience forms only a small part of an individual's total experience — experience accumulated 
over 16 waking hours a day, 365 days a year, each year of their life. For someone 45 years of 
age, the impact of a one-week (40-hour) training experience would be: 



40 „ 40 = .00015 of the trainee's 

(16) x (365) x (45) 262,800 total life experience 

The training fraction attempts to make two points: 

1 . Do not be overly optimistic about what any single training experience can produce in the 
way of behavioral change. (In the above example, the 40 hours devoted to training represented 
only .00015 of the trainee's total life experience.) 

2. The training experience must be so meaningful that it exerts an impact which overcomes its 
relatively brief timespan. 



Hours Devoted to Training 
(16) x (365) x (Age in Years) 



ORGANISATION CHART 

Training Department, International Centre for Advanced Technical and Vocational Training 
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JUBAIL TRAINING CENTER 
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TRAINING POLICY ANALYSIS 


1 . 


Axioms 


2. 


Key terms 


3. 


A definition 


4. 


What to look for 


5. 


Relevance, effectiveness, and efficiency 


6. 


Basic steps 


7. 


Financial options 


8. 


Study the demand side 


9. 


Demand-side issues 


10. 


Study the supply side 


11. 


Supply-side issues 


12. 


Gathering information 


13. 


Training questions 


14. 


Comparing education and training 



Note. Parts of this work are based on personal communications with W. Durr, International 
Labour Office, Geneva, Switzerland. 



Clifton P. Campbell 
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Axioms 



Societies seek economic and social 
development. 



Development depends on, among other 
things, people with useful knowledge, 
skills, and attitudes — and therefore on 
training. 



Countries need to formulate policies, 
including training policies, to meet 
economic and social development 
objectives. 



Three training policy ingredients: 

1 . Program orientation 

2. Resource allocation 

3. Institutional arrangements 



Policy formulation is one thing; 
implementation is another. Realistic 
polices must therefore recognize a variety 
of constraints; moreover, they must 
indicate priorities. 



Key Terms 



Policy : 

A course of action by which it is proposed to 
achieve a certain objective, notably in government. 



Development : 

A process of economic and social change typically 
fueled by income growth. 



Training : 

An effort organized to transfer knowledge, skills, 
and attitudes which people need in order to be 
productive or to change their working behavior. 



System : 

A set of things, parts, ideas, that work together or 
form a whole, e.g. training policies, programs, 
resources, and institutions. 



Analysis : 

Separation into parts, e.g. the unraveling of a 
system to know how it works or what is wrong 
with it. 
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A Definition 



Training Policy Analysis is a process of 
inquiry aimed at recommending how 
training delivery systems can be made 
more (a) relevant, (b) effective, and 
(c) efficient. 




in other words, 



Training Policy Analysis is an intellectual 
effort to decide: 

where we stand; what is the problem; what 
is the solution; where we want to go; how 
best to get there given the circumstances, 
context, environment, i.e. the constraints 
and opportunities, the distance, time at our 
disposal, and given other, wider objectives. 
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What To Look For 
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Training Policy Analysis: Relevance, Effectiveness, and Efficiency 
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Training Policy Analysis: Basic Steps 




Training Policy Analysis: Financial Options 
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STUDY THE DEMAND SIDE 



Q: What should the training system do? For 
whom? 

1 . Recognize the national context 

2. Identify current national training objectives 

3 . Summarize current development objectives 

4. Establish current and future labor market 
demand (need) for skilled workers 

5 . Establish social demand for training 
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DEMAND SIDE ISSUES 

1. Demand, need, opportunity, feasibility 

2. Appropriate and inappropriate training 
objectives 

3. How specific can one get, should one get 
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STUDY THE SUPPLY SIDE 



Q: What is the training system? What does it 
do? What is it capable of doing? 

1 . Size of the system 

2. Breadth and depth of coverage 

3 . Standards and pass rates 

4. Training costs 

5 . Personnel 

6. Facilities, equipment, materials 
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SUPPLY SIDE ISSUES 



1 . Division of labor 

2 . Capacity 

3. Capability 

4 . T argeting 

5. Trainability 

6 . Standards & testing 

7 . Follow-up 
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Gathering Information 



on the demand side : 

1 . Workforce Projection and Forecasting 
Approaches 

■ Manpower requirements approach 

■ Econometric models 

■ Input-output models 

2. Labor Market Signalling Approaches 

■ Reporting by employment 
services 

■ Employer surveys 

■ Key informant interviews 

■ Analysis of job advertisements 

■ Follow-up studies 



on the supply side : 

■ Enrollment data 

■ School visits 

■ Follow-up studies 

■ Labor turnover surveys 
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Training Questions 



WHY 

should people be trained? 



WHO 

should be trained? How many? Who 
should be trained first? 



HOW 

is training best organized, managed, and 
financed? Who does what? 



WHAT 

should people learn? How is this 
determined and assessed? How about 
standards? 



HOW 

is training best delivered? By whom? 
Where? When? 
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Human Resource Development: Comparing Education and Training 
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SELECTION AND DEVELOPMENT OF THE TRAINING STAFF 



Directions : Each of the following items contains four (4) alternatives, one of which provides the 



best answer. Circle the letter which precedes the best answer. 



1 . When an organization selects internal candidates 
to join the training staff, managers should 

A. select only those who are subject matter 
experts with long experience. 

B. invest the resources needed for new trainers 
to develop essential job skills. 

C. choose those who need no further training to 
perform their new duties. 

D. make sure first that no qualified people are 
available from outside the organization. 

2. The first step in filling a training position is to 

A. publicize the vacancy. 

B. inform employment agencies of the opening. 

C. contact professional organizations. 

D. specify the job requirements. 

3. When a candidate selected for a training position 
is deficient in knowledge, skills, or other 
selection criteria, the best thing for the training 
manager to do is 

A. invest in the necessary training and 
development. 

B. redefine the job specifications. 

C. change the selection criteria for future 
positions. 

D. reduce the job performance requirements. 

4. Developing career paths for members of the 
training staff should 

A. have as little effect on organizational goals as 
possible. 

B. be based entirely on the needs of the 
organization. 

C. combine the needs of the organization with 
the personal growth goals of employees. 

D. mainly be left up to the individual employees. 
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COMPETENCY ANALYSIS FOR TRAINERS 



Table 1 COMPETENCY/FUNCTION 



TRAINING AND DEVELOPMENT FUNCTION 



COMPETENCY AREA 


INSTRUCTOR 


DESIGNER 


MANAGER 


CONSULTANT 


INDEX 


1. ADMINISTRATION 


o 


o 


• 


€ 


7 


2. COMMUNICATIONS 


• 


• 


• 


• 


10 


• 3. COURSE DESIGN 


€ 


• 


O 


• 


12 


4. EVALUATION 


• 


• 


• 


• 


14 


5. GROUP DYNAMICS/PROCESS 


• 


• 


o 


• 


16 


• 6. INSTRUCTIONAL TECHNIQUES 


• 


€ 


o 


• 


18 


7. LEARNING THEORY 


€ 


• 


o 


# 


23 


8. MANPOWER PLANNING 


N/A 


N/A 


€ 


€ 


25 


9. PERSON/ORGANIZATION INTERFACE 


€ 


o 


• 


• 


27 


10. RESEARCH & DEVELOPMENT 


o 


• 


o 


• 


29 


11. TRAINING EQUIPMENT & MATERIALS 


* 


- • 


o 


9 


31 


12. TRAINING-NEEDS ANALYSIS 


o 


€ 


* 


<§ 


35 



PROFICIENCY LEVELS: 

• HIGH [mustl level C MEDIUM [want] level O LOW level N/A Not Applicable 



In the detailed “Area of Competency” skill lists within the main part of the document, only the highest level is 
designated. This is the “must” or "survival” skill which is absolutely essential to performance success. While the 
absolute requirements can and will vary from position to position, those identified as “survival” needs will be found 
to be correct in the majority of cases. 

HOW TO USE THIS DOCUMENT 



The following steps indicate how to make the most effective use of Competency Analysis: 

1 . Select the T & D function to be analysed from Table 1 (see above) 

2. Reading down the Competency Area column, identify the particular competency area to be studied. Note the 
page number under the Index column and turn to that page for the detailed list of competencies. 



3. Review the detailed list of types of knowledge and skills 1 for the particular competency area selected. Of 
greatest importance are those items identified as "survival" needs (•) beneath the column representing your 
particular T& D role. 2 

4. Consider each skill listed with regard to whether or not it is required in your own unique position or 
organization. If the skill is not needed, enter an X in the column headed N/A (not applicable). 3 

5. Now consider any unique skills or knowledge areas which are demanded by your position or company and 
which are not included ir. the list. Enter these skills on the blank lines provided at the bottom of each list. 4 Be 
sure to indicate the proficiency level for any skills added. 

6. There are two competency rating columns at the right side of the page. The first of these is titled PERSON 5 
and is to be used to rate the competency skill level of the incumbent of the position under analysis. Consider 
each skill and rate the incumbent accordingly. Use a scale of 0 (no competence) to 5 (high competence). 

7. The second column at the right is titled JOB 6 and relates to the significance of the skill for the successful 
execution of the job to be analysed. Use the same 0 to 5 scale in this column. 

8. Once both the person and the job have been rated, subtract the rating under JOB from the rating under 
PERSON and plot the variance in the PERSON/JOB VARIANCE column. 7 Be sure to maintain consistency in 
calculating the variances; that is, always subtract the job rating from that given the person. Because the scale 
is calibrated to both a plus ( + ) and a minus (-) extreme, individual strengths and weaknesses may be visually 
identified by simply noting the plotted results on the appropriate side of the scale. 
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WHO SHOULD US E THIS DOCUMENT . 

Because of its flexibility and broad application. Competency Analysis will be useful across the entire T & D 
spectrum. 

• NOVICE PRACTITIONERS Those who are new to the T & D role can use Competency Analysis to assess 
their performance readiness and to establish personal development plans. 

• EXPERIENCED PRACTITIONERS Those who are settled into their particular position may employ 
Competency Analysis to further enhance their skills, monitor their performance, and plan their careers. 

• CAREER PROFESSIONALS Established T & D professionals may employ Competency Analysis for 
self-evaluation and career planning. The document also provides them with a constant checklist against which 
to judge their knowledge and skills with respect to changing demands or emphases of job performance. 

• MANAGERS Those who supervise T & D professionals may employ Competency Analysis to recruit, evaluate, 
coach, and counsel subordinates. By using the document to reach mutual agreement about criteria for 
measuring job performance, evaluation and counselling sessions will become more productive and less likely to 
develop into adversarial confrontations. 

• T & D ASPIRANTS Individuals who seek careers in T & D will find Competency Analysis useful for assessing 
their opportunities and preparing for their entry into the professional stream. 

• COURSE DESIGNERS Those who are designing programs for the instruction of T & D practitioners may use 
Competency Analysis to ensure appropriate content and subject emphasis. 



APPLICATIONS 



Competency Analysis offers a variety of applications which, of course, may be modified according to individual 
need. 

• CANDIDATE SELECTION MODEL Using Table 1 (p. 2), personnel officers can establish position models 
against which available candidates for a T & D job can be compared. This model can be expanded to a full 
position profile by referring to the corresponding detailed knowledge/skill lists. (Use steps 1 to 5 under “How to 
Use This Document”.) 

• SELF-ASSESSMENT An individual may assess his or her knowledge/skill level against current position 
criteria. (Use steps 1 to 8 under "How to Use This Document”.) 

• DEVELOPMENT PLANNING From the method of assessment described above, a specific development plan 
may be established to address knowledge/skill shortfalls. 

• PERFORMANCE TRACKING Repeated use of this method of assessment will reveal the amount of progress 
or improvement achieved over time. 

• PERFORMANCE EVALUATION Managers of T & D professionals may evaluate performance through the 
assessment approach described above. Areas of knowledge/skill requiring coaching and/or counselling will 
become readily apparent. 

• POSITION/JOB DESCRIPTIONS Accurate and detailed job descriptions will result from employing the model 
described above. 

• CAREER PLANNING A detailed assessment, as described in “Self-Assessment" above, will reveal individual 
strengths and areas needing improvement. Comparison of the results with the criteria for other T & D functions 
identifies the areas of knowledge/skill that require development and enables the individual to develop a realistic 
plan for advancing in his or her career. (See "Personal Planning Guide” at the end of this document.) 

• CURRICULUM OR COURSE CONTENT By employing steps 1 to 5 under “How to Use This Document”, a 
course designer may readily identify the required subject content in a training program for T & D practitioners. If 
pre-course testing takes place, the course designer can carry the process through to step 8 to determine which 

• subjects require major emphasis and which can be dealt with briefly. 



DIAGNOSTIC COMMENTS 



The following may prove helpful in interpreting the variances obtained after rating both the job and the job 
incumbent: 



• When the job rating exceeds that of the person rating by 1 and the person rating is 2 or more, the shortfall is not 
serious and can be given low priority in the resulting development plan. 



• When the job rating exceeds that of the person rating by 1 and the person rating is 0 or 1 , the skill should receive 
moderate to high priority depending upon the seriousness of other skill shortfalls. 



O When the job rating exceeds that of the person rating by 2 and the person rating is 2 or 3, the skill should receive 
I RI ( moderate priority in any resulting development plan. 
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• When the job rating is 3 or more points higher than the person rating, the shortfall is serious and should receive 
priority attention. 

• Any variance favouring the job where the person rating is 0 is serious and should receive priority attention in any 
resulting development plan. 

• When the person rating exceeds that of the job by 3 or more, the person should be alert to a possible resulting 
performance problem. Individuals tend to rely upon those skills in which they are proficient to the extent that 
over-use has a negative effect. Any T & D professional who, upon using Competency Analysis, discovers highly 
rated skills which appear to be of little or no value to the job should be cautioned against their excessive use. 
Very often, one’s most severe performance drawbacks are merely the result of one's overdeveloped strengths. 

• When the amount by which the job rating exceeds or falls below the person rating is erratic - that is, ranges 
widely in the degrees of variance - the person should concentrate his or her development plan upon achieving 
some kind of consistency. Failure to do this may result either in a performance in which weaknesses seem 
greater when juxtaposed with major strengths, or in awkwardness as a result of the person’s attempts to avoid 
using weak skills entirely. If Competency Analysis has been completed fairly and objectively and a skill is 
identified as essential to the job, its use cannot be avoided. Balanced performance from aT& D professional is 
far superior to a performance highlighted by erratic highs and lows. 



It is extremely difficult for individuals to be entirely objective about themselves 
or their jobs. Because of this, it is recommended that the skill analysis be 
carried out by two people: the individual being analyzed and, if possible, his 

or her direct superior. Points of major difference can be discussed for agreement, 
or it may be agreed at the outset that an average of the two ratings will be 
taken. This practice produces more objective results and helps to clarify the 
performance expectations of the two people involved. 



Source: 

Ontario Society for Training and Development. (1984). Competency analysis for 
trainers: a personal planning guide . Toronto, Ontario, Canada: Author. 
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Human Resource Development Department 



The University of Tennessee 



DETERMINING THE DEMAND FOR SKILLED WORKERS 



Learning Objectives 

Upon successful completion of this module, you will be able to: 

1. Name and describe important aspects of three workforce projection and forecasting 
approaches 

2. Differentiate among the workforce projection and forecasting approaches 

3. Name and describe important aspects of five labor market signalling approaches 

4. Differentiate among the labor market signalling approaches 

5. Assess inputs and judge the value of the information provided by the different 
approaches for determining skilled worker demands 

6. Prepare a questionnaire for use in a follow-up study 

7. Pilot-test the questionnaire on a sample of trainees and make all appropriate revisions 
Rationale (purpose) 

To provide information on different approaches used in determining labor market demand 
(number of workers needed with particular skills). This demand information is important for 
planning vocational education and training (VET) that is relevant to the skilled worker needs of 
the local, regional, and national economy. 

Introduction 

This module provides an overview of the main methods and techniques relied upon to 
determine the demand for skilled workers. These are the (a) workforce projection and 
forecasting approaches, and (b) labor market signalling approaches. 

It is widely agreed that VET opportunities ought to closely match the need for workers 
with particular skills. However, current and future skilled worker needs are not easily and 
precisely determined. Profiles of job and skill requirements are a consequence of the changes 
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taking place in technology, work organization, consumer patterns, and the restructuring of the 
world-wide economic system (Campbell, 1996, chap. 1). 

Even though experts question the utility and validity of projecting or forecasting long-term 
national skilled workforce demands in a free-market economy, public agencies and research 
institutions as well as private consulting firms continue the practice. This is largely because 
decisions need to be made about the utilization of existing and the availability of new training 
facilities and faculty, the number of trainees to be admitted to different courses/programs, and 
so forth, in advance of labor market entry by those who are to be trained. The longer the 
training period, the greater the need to plan ahead of labor market demand (Lauglo, 1993, p. 

«)• 

In response to criticism, projections and forecasts are now largely confined to a sectoral or 
regional level, where they are more apt to provide useful information. The trend is toward 
collecting more valid and reliable information for short-term projections and forecasts, which 
tend to be more accurate, and making adjustments based on routine monitoring through 
generally accepted labor market signalling approaches. 

Organization of the module. This module is organized in three parts. The first part 
presents workforce projection and forecasting approaches. Three approaches are briefly 
described and the salient advantages and disadvantages are pointed out in text and listed in 
Table 1. In addition, three other approaches are identified. Table 1 is followed by a progress 
check, to facilitate your understanding of the content covered and to promote learning. 

Part 2 of the module exhibits five labor market signalling approaches. A description is 
provided for each approach and the advantages and disadvantages are pointed out in text and 
listed in Table 2. This is followed by Table 3, which rates each approach against the maximum 
potential value of the various categories of information it collects. Like Part 1, there is a 
progress check, to facilitate a review of the content covered in order to promote learning. 

Part 3 begins with a comment — a series of remarks and observations pertaining to Parts 1 
and 2 of this module. This is followed by three supplemental instruments which provide 
information, examples, and guidance relative to determining workforce demand. They are 
titled: 
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1. An example of the organization and analysis of key informant interviews 

2. Guidelines for conducting follow-up studies 

3. A prototype follow-up questionnaire 

Progress checks 3A and 3B are included in this last part of the module. They are intended to 
facilitate your understanding of significant content and promote learning. 

A note on terminology. It is important at this point to clarify three often-used words: 
projection, forecasting, and planning. Each of these words has a specific meaning; nevertheless, 
the words projection and forecast are often used interchangeably in the literature. 

Projection is a protraction in the future of a past trend, in accordance with assumptions of 
extrapolation or deviation. The approaches used are mechanical and are not based on any 
explicit formulation of how labor markets function. Projections are not forecasts unless they 
involve a probability. 

Forecasting is to make an assessment of what will happen between the present (base year) 
and a future date (target year). The time period is a major consideration. Assessments are 
usually quantified, made with a certain degree of confidence in their probability, and subject to 
assumptions concerning the nature and direction of future trends. Assumptions are the 
backbone of the forecast. 

Planning consists of conceiving a desired future and mobilizing the means to achieve it 
(Bertrand, 1992, chap. 7). For our purposes here, planning means anticipating and taking 
corrective action to avoid workforce imbalances (shortages and surpluses), based on the best 
possible information about current and future labor market demand. Thus, planning aids in the 
intelligent and effective preparation of human resources (Campbell, 1996, chap. 1). 
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Part 1 — Workforce Projection and Forecasting Approaches 

The three workforce projection and forecasting approaches described here are the (a) 
manpower (workforce) requirements approach, (b) econometric models, and (c) input-output 
models. All three attempt to quantify the long-term future workforce demands of the economy. 

Manpower (Workforce) Requirements Approach 

The workforce requirements approach relates forecast or planned economic growth to the 
output of VET institutions (schools, institutes, colleges, apprenticeships, etc.) based on a 
number of plausible assumptions. A general assumption is that certain sectoral economic 
growth rates are reachable targets (Lauglo, 1993, p. 7). The workforce requirements derived 
from these targets can be used in planning the output of vocational institutions. 

This approach, as usually practiced, has the following five main steps: 

1. Forecast employment levels in the target year (some future date) for each economic 
sector/subsector 

2. Estimate the distribution of various occupations (staffing patterns) in each 
sector/subsector 

3. Convert the employment forecasts into a set of projections by occupation 

4. Project occupational replacement needs — those arising from job turnover 
(retirements, disabilities, deaths, promotions, and occupational mobility) 

5. Combine steps three and four for each occupation to estimate job (employment) 
possibilities 

Advantages. The methodology is straightforward, transparent, and appeals to common 
sense. It is readily comprehensible and has remained popular with economists and policy- 
makers. Data requirements change according to variations of the model with regard to 
different levels of aggregation and detail. On the whole, however, data requirements are 
relatively modest. This approach provides long-term quantitative skilled worker requirements 
(Campbell, 1996, chap. 1). 

Disadvantages. A set of evaluations of workforce requirements studies concluded that, 
while employment forecasts at the sectoral level (step 1 in the previous list) have been 
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reasonably accurate, their conversion to occupations (step 3) is problematic (Hinchliffe, 1993). 
Furthermore, the reliability of this approach depends essentially on the accuracy of the plausible 
assumptions. Unfortunately, experience has shown that reliability remains an elusive goal; the 
discrepancy between actual and forecast workforce requirements is usually substantial (Mingat 
& Tan, 1988, p. 107). The reasons include weaknesses caused by overly-optimistic estimates of 
employment growth as well as increasingly rapid economic change. Other weaknesses are the 
frequent lack of allowance for technological advances, changing business practices, occupational 
mobility (e.g., resignations, layoffs, and discharges), and withdrawals from and returns to the 
workforce (e.g., quit to raise a child or pursue further education, then return). Additionally, 
jobs are often filled by those who lack the skills theoretically required but who will work for 
less. 

As a rule, accuracy and reliability diminish with longer-term projections and with greater 
disaggregation, particularly greater occupational detail (Bertrand, 1992, chap. 1; Lauglo, 1993, 
p. 7). This dilemma is at the root of a general dissatisfaction by VET planners who insist that 
accurate training needs assessment requires detailed — rather than macro-type — projections. 

Econometric Models 

These models, which have undergone refinements since the 1960’s, represent a set of 
equations describing the complex interrelationships of the different economic sectors. 
Econometric studies are a sophisticated long-term forecasting approach customarily conducted 
by economists. Their output is a set of estimates of levels of employment in various sectors for 
the target year (Bertrand, 1992, chap. 1). Users of this information are cautioned not to 
assume a straight-line trend between the base year (year of the study) and the target year. 
Econometric models are an alternative to the first step of the workforce requirements approach. 

Advantages. This approach is sensitive to a large variety of factors affecting the level and 
structure of employment, taking into account indirect and local inter-sectoral effects. 
Methodological improvements allow for replacement needs to compensate for retirements, 
deaths, occupational mobility, and so forth. 

Disadvantages. Data requirements, as well as model and data base maintenance costs, are 
considerable. In addition, due to the technical complexity of the model’s documentation, the 
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techniques applied pose problems of accessibility and comprehensibility to VET planners and 
policy-makers. Moreover, the data are national and, therefore, not entirely useful for regional 
or local planning. 

Accuracy is also a problem with models which are based on projections of population, 
workforce, productivity, consumption, and overall output. This is because of the difficulty in 
forecasting economic activity, technological change related to productivity, and specific needs, 
which change due to labor and capital mobility in given market areas (American Institutes for 
Research, 1976, p. 45). 

Input-Output Models 

Like the econometric models just described, input-output models constitute the first step of 
the workforce requirements approach. Input-output models are used to construct a table that 
displays the exchange of goods and services among producing and purchasing industries over a 
set period of time. The industrial output is translated into employment demands within 
industries. Total employment within an industry is related to its total output and expressed in 
terms of a workforce input coefficient. These coefficients show the workforce requirements of 
an industry per unit of output (Bezdek, 1974, p. 3-13). 

The following three steps characterize the input-output analysis: 

1. A projection is made of the output of a particular industry. For example, mining — 
the tons of coal mined over the period of time desired. 

2. A workforce coefficient — for example, one miner per 10,000 units of output — is 
applied to the absolute increase in production to arrive at an estimate of workforce 
requirements. 

3. The projection and workforce coefficient are then translated into VET requirements 
(Mingat & Tan, 1988, p. 104). 

Advantages. Input-output models are capable of producing highly detailed employment 
projections, taking into account inter-industry relationships. The models are equally appropriate 
for impact analysis (e.g., the effects of government programs on a regional economy). 

Disadvantages. Development of a national input-output table is a complex, laborious, and 
expensive undertaking. First, a vast amount of quantitative information must be gathered from 
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many and varied sources and then ordered. Next, an enormous amount of statistical 
compilation, estimation, correction, balancing, and reconciliation must be completed. 
Consequently, only organizations with specialists who have access to sophisticated computer 
capabilities can construct input-output tables. 

Other Approaches 

There are other workforce projection and forecasting approaches which are more or less 
appropriate to assessing future labor market demand. These include the normative methods 
which specify desired ratios (e.g., the number of practical nurses to registered nurses, 
apprentices to journeymen, etc.), the international comparison method (borrowing historical 
projection data from countries at more advanced levels of development), and the rate of return 
analysis method (instead of stipulating skilled worker requirements, it provides indicators for a 
necessaiy expansion or contraction of certain training courses and/or programs). The rate of 
return analysis method (model) also takes into account the cost of producing a given number of 
graduates. 

Detailed descriptions of these three approaches, as well as the three previously presented, 
can be found in Training Needs Assessment and Monitoring by Lothar Richter (pp. 31-39), 
which also provides further references. 



Note. The advantages and disadvantages of three workforce projection and 
forecasting approaches can be easily compared by consulting Table 1 on the 
following page. 
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Table 1. Advantages and Disadvantages of Workforce Projection and Forecasting Approaches 



Advantages 



Disadvantages 



Manpower (workforce) requirements approach 



• Straightforward and transparent 
methodology appeals to common sense 

• Readily comprehensible 

• Popular with economists and policy-makers 

• Relatively modest data requirements 

• Provides long-term quantitative skilled 
worker requirements 



• Conversion of sectoral-level forecasts to 
occupations is problematic 

• Reliability affected by uncontrollable factors, 
such as overly optimistic estimates of 
employment growth, rapid economic change, 
lack of allowance for technological advances, 
changing business practices, occupational 
mobility, and withdrawals from and returns to 
the workforce 

• Accuracy and reliability diminish with longer- 
term projections and with greater 
disaggregation 



Econometric models 



• Sensitive to a large variety of factors which 
affect the level and structure of employment 

• Methodological improvements allow for 
replacement needs to compensate for 
retirements, deaths, occupational mobility, 
etc. 



• Extensive data requirements 

• High data maintenance costs 

• Highly technical and, therefore, difficult for 
vocational training planners and policy- 
makers to access, comprehend, or use 

• National data not entirely useful for local or 
regional planning 

• Accuracy problem with models based on 
population, workforce, productivity, 
consumption, and overall output projections 



Input-output models 



• Capable of producing highly detailed 
employment projections 

• Equally appropriate for short-term 
forecasting and impact analysis 



• Complex, laborious, and expensive 

• Extensive data collection and manipulation 
requirements 

• Requires specialists and sophisticated 
computer capabilities 
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Progress Check -- Part 1 — Workforce Projection and Forecasting Approaches 

Directions. Read each of the following items carefully and write in your answer. If you have 
difficulty with any of the items, go back and review the relevant content before continuing. Check 
your answers against those provided in the Progress Check Feedback at the end of this module. 

1. List the three workforce projection and forecasting approaches described in Part 1. 

A. 

B. 

C. 

2. Identify the approach that relates forecast economic growth to the output of VET 

institutions. 

3. Name the reasons for the difficulty in accurately converting from sectoral/sub-sectoral to 
more specific occupational employment possibilities in the workforce requirements 
approach. 

A. 

B. 

C. 

D. 

E. 

F. 
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Progress Check -- Part 1 (continued) 

4. Identify two models that provide an alternative to the first step of the workforce requirements 
approach. 

A. 

B. 

5. Name the workforce projection and forecasting approach that takes into account inter- 
industry relationships. i 

6. Name the steps which characterize an input-output analysis. 

A. 

B. 

C. 
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Part 2 — Labor Market Signalling Approaches 



Rather than depend on sophisticated long-term projections and forecasts, with their 
difficulties and shortcomings, vocational training planners need to consider the merits of various 
labor market signalling approaches. The five approaches illustrated in Figure 1 involve the 
frequent collection and careful analysis of data on workforce supply and demand in 
local/regional labor markets. Labor market signalling approaches identify current and short- 
term (future) job opportunities and can provide information on knowledge and skill 
requirements as well as employment practices. 




Public Employment Services 

Labor market signals can be captured by monitoring workforce movements in the 
employment and unemployment (layoffs and resignations) of workers. Most public (state) 
employment services collect and compile information on unemployment rates, job openings, and 
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placements. Nevertheless, the data is seldom analyzed and few, if any, decisions are reached 
concerning present workforce requirements and labor supply or probable future requirements. 

For example, tallying the number of openings, by job title, is not only useful in itself, but 
makes it possible (over time) to monitor trends in the demand for and supply of skilled 
workers. Thus, when job opening rates fall, unemployment rates rise, or employment growth 
declines for workers with particular skills, this signals a downturn in the need for these skills 
and, hence, in the benefits of providing training to produce them. There is a great and growing 
need for the information a public employment service could supply. However, such information 
must be available in a usable form (Campbell, 1996, chap. 1). 

Advantages. Public employment services act as an intermediary between people seeking 
jobs and employers looking for qualified workers. Consequently, they are in touch with labor 
market happenings at local, regional, and state levels. As a result, they are in a position to (a) 
collect, compile, analyze, and report current information on job openings and unemployment 
rates; and (b) project future replacement and growth openings. This information is valuable to 
vocational training planners, who can use it in determining the need for training in a particular 
job or occupation. The analysis and dissemination of this information does not require 
investment of considerable new resources, but mainly requires broadening the focus to include 
labor market information analysis. 

Disadvantages. Since the registration of people for unemployment benefits and the 
notification of vacancies (job order requests) by employers do not cover the whole of the 
supply/demand situation, the data available from public employment services is more or less 
incomplete. Furthermore, the data may not be representative of the overall labor market 
situation. 

While details are not available on the extent to which employers report job openings to the 
local public employment service, both the number of employers reporting and the percentage of 
available jobs reported are considered to be low. Additionally, the jobs reported tend to be 
those with lower level qualification requirements. In some instances, these jobs also appear in 
newspaper help-wanted ads. Thus, the information available is inadequate and could be 
misleading. 
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Apart from this, public employment services are more highly concentrated in urban areas 
where they serve the formal (wage-earning) sector. Urban offices appear to function largely as 
a job exchange for the unemployed, with unemployment benefit claims taking up much of their 
time. Any generation and dissemination of labor market information is performed as a side- 
line. 

In other words, the full potential of public employment services to determine and report 
the present and future demand for skilled workers is far from being adequately developed. 
Notwithstanding this situation — on perhaps on its very account — there is often little or no 
cooperation between vocational training planners and public employment services, especially at 
the local level. 

Job Advertisements 

There is little or no systematic monitoring of job (help-wanted) advertisements that appear 
in newspapers, trade publications, professional journals/magazines, and other print media. Yet, 
the collection and analysis of these public listings provides current labor market demand 
information and, over time, reveals employment trends in a host of occupations, e.g., a decrease 
in the demand for surveyors and an increase in the number of job vacancies for cooks. 

Help-wanted ads provide a great body of handy and usable information. Unfortunately, 
only a part of the actual job openings are advertised and the information on these is often 
imprecise and/or incomplete. Therefore, the employer, employment agency, or service must be 
contacted in order to obtain further details, such as (a) whether the job was newly created or a 
replacement, (b) the number of applicants, (c) applicant qualifications relative to knowledge and 
skills requirements, and (d) the occupational profile of those hired. Skill gaps on the part of 
applicants point out a training need. 

Advantages. When employers determine that classified ads are a cost-effective way of 
attracting a large pool of qualified job seekers, they advertise in newspapers, especially the 
Sunday editions. Large circulation and big city newspapers often contain voluminous sections of 
job advertisements which literally go begging for analysis. Such analysis could be undertaken by 
vocational training planners who, in discussing ads with the employers who floated them, could 
benefit directly from the views and attitudes of those employers regarding the knowledge and 
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skills required. The analysis results would at least complement the signals received from other 
sources such as employment services. The collection and analysis of job advertisements is a 
comparatively easy and inexpensive approach to identifying jobs with favorable employment 
prospects. 

Disadvantages. The jobs advertised are principally located in metropolitan areas. They 
often include more of the white- than blue-collar portion of the job market and may be limited 
to certain sectors of the economy where there is a high turnover rate (viz., hospitality and 
tourism, wholesale and retail trade, and construction). In addition, many employers do not use 
want ads for a host of reasons (e.g., they employ union workers from a hiring hall), and those 
that do tend to be the bigger companies who may also list their job vacancies with the public 
employment service. Consequently, the variety and number of job openings is under- 
represented. A further complication is the tendency of some employment agencies (placement 
firms) and employers, especially in the service industry, to run ads for jobs that have been filled 
or otherwise do not currently exist. This is done in order to expand their applicant pool in 
anticipation of future openings. Apart from these disadvantages, the number of advertisements 
in local or regional newspapers may be insufficient for determining demand/supply imbalances 
and trends. 

Moreover, this approach deals with past and present labor market demand, not the future 
needs of a locality/region. It cannot, for example, indicate employment needs based on the 
growth of existing companies or the arrival of new ones. Furthermore, labor market 
information emanating solely from the analysis of job advertisements is inconclusive and must 
be compared with data from other approaches. This is essential in order to accurately 
determine what training courses and programs are necessary to supply the skilled workforce 
needed by the economy. 

Key Informant Interviews 

Interviews of individuals who are knowledgeable on workforce demand and the supply of 
skilled workers in a geographic area or economic sector offer another way to obtain valuable 
labor market information. Essential preconditions for the satisfactory outcome of key informant 
interviews include the careful selection of experts from (a) local/regional businesses; (b) public 
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agencies; (c) economic development authorities; and (d) employer, worker, and professional 
organizations. Other important preconditions are the use of a structured interview schedule 
limited to well-constructed core questions, the timely analysis of the information obtained, and 
regular and frequent follow-up of the signals captured. 

Core questions focus on current and expected workforce supply and demand imbalances. 
They also seek to determine factors underlying such imbalances, and the most critical workforce 
issues for the next 5 years. Other questions can address employer recruitment, screening, 
hiring, promotion, and training practices. 

It is important to add that regular consultations with key informants, to discuss workforce 
supply and demand problems and/or interpret demand projections resulting from other 
approaches, are a prudent practice. Experience has shown that carefully selected individuals are 
able and willing to provide input. 

Advantages. Key informant interviews are straightforward and modest in cost. They yield 
current indicators of workforce supply and demand imbalances in local/regional labor markets, 
and give early warning signals about forthcoming changes. In addition, information can be 
gathered on the various screening, hiring, and training practices in use. For example, are skilled 
jobs filled by promoting and training semi-skilled employees, or are apprenticeship programs 
used to prepare skilled workers? If these or other screening practices are used, there may be 
little opportunity for applicants who have already completed a training course or program and, 
therefore, expect skilled-worker jobs and wages (Campbell, 1996, chap. 1). 

Key informant interviews can be held with individuals and groups, conducted in both the 
public and private sectors, and used in formal as well as informal labor markets. Selection of 
the proper key informants (those who have the facts) may be based on brief discussions, cross- 
checked by the question: Whom do you consider most knowledgeable on workforce demand in 
this area? In addition to others, the managers of private temporary placement (staffing) 
agencies can provide useful, albeit incomplete, information. Agencies that specialize in a 
particular occupation, such as clerical workers, often monitor supply and demand trends for 
their own business purposes. While they can provide some useful information, private agencies 
tend to know more about the demand for experienced workers than for new graduates. 
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Conduct of interviews and analysis of information collected can be entrusted to 
employment security, labor, and other state or local government departments, research 
institutes, universities, and, last but not least, VET providers. 

Disadvantages. Key informants provide mainly qualitative information which reflects their 
personal views. In addition, the interviewer’s bias, demeanor, and appearance can distort 
responses. Consequently, the information collected must be weighed with other available data. 
Apart from this, informants will not be representative in terms of research rigor, since the 
primary selection criterion for a key informant is "knowledge" (and not "randomness"). 

Whether these disadvantages are significant depends largely on the kind and scope of 
workforce information which VET providers have agreed as being essential for the effective and 
efficient planning of training courses and programs. 

Employer Surveys 

Ultimately, employers are the ones who decide on hiring. Theirs is the last word on what 
the market wants; they are the market. It makes sense, therefore, to seek their input. The idea 
is neither new nor original. The problem is how to do it. The most straightforward way is to 
survey private and public sector employers directly. 

Mailed questionnaires and personal interviews are widely used methods of collecting 
information. Questionnaires and interview schedules can vary from open-ended to closed-ended 
instruments. The closed-form instrument is recommended when categorized data is needed; 
whereas, the open-ended is best-suited for preliminary exploration of untried situations. 
Telephone interviews are limited in use and not usually appropriate for a comprehensive survey. 

Large, medium, and small businesses within a geographic area (including all places that 
workers are willing to travel without changing their residence) are identified and asked to 
estimate their present and future workforce requirements. They are also asked to provide 
information on knowledge and skill requirements, as well as any constraints in hiring VET 
graduates. This information helps to identify which businesses and occupations are growing and 
which are declining. It augments the data gathered from other approaches, such as reporting by 
employment services (Campbell, 1996, chap. 1). 
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Advantages. Employer surveys go right to the source of labor market demand, and can 
provide valuable information in a reasonably short time. By directly involving employers in 
needs determination and by considering their views regarding occupational projections and 
forecasts derived from other approaches, employers’ willingness to cooperate in providing 
information is increased (American Institutes, 1976, p. 44). 

The personal interview method of data collection provides a high response rate and yields 
precise and complete information on a host of relevant questions. The interview can also be 
used to inquire about anticipated changes in technology and to inform employers about the 
advantages of hiring VET graduates. 

Disadvantages. Not all employers are able (or willing) to provide detailed predictions of 
workforce demand, especially those needs beyond the immediate future (not much more than 
one year). This is because many employers don’t gather the data necessary for reliable 
estimates. In addition, when estimates are provided, they are apt to be biased toward 
overestimation if the employers feel that their replies will positively affect the availability of 
skilled workers. For example, if the skills concerned are scarce, employers will tend to 
overestimate their needs (Bertrand, 1992, chap. 2). 

Furthermore, different ways of classifying jobs by various employers have detrimental 
effects on the comparability of the information collected. Comparability also suffers as changes 
in the workplace alter job requirements, especially when the jobs are not reclassified. 

Moreover, surveys that ask employers to provide relatively long-range estimates may not provide 
valid or reliable data. This is especially true during periods affected by extreme structural 
change and/or severe economic fluctuations. 

Employers are deluged with forms and questionnaires to be filled out. Consequently, they 
respond under duress, even to government agencies. Suffice it to say, returns will be 
disappointingly few and the information provided may not be complete or adequate for 
planning purposes. The personal interview approach yields better results. 

While surveying employers may not provide valid or reliable quantitative information on 
labor market demand, it is, nevertheless, an essential component of the qualitative analysis of 
the content and evolution of jobs and an assessment of labor market functioning (Bertrand, 
1992, p. 34). 
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A further drawback of this approach is that costs vary from reasonable to high levels 
depending on the (a) data collection method used (mailed questionnaire or personal interview), 
and (b) number of employers surveyed. In any case, the main problem is that employer surveys 
do not cover skilled worker demand in the informal (self-employed) sector, particularly in rural 
areas. 

Follow-up Studies 

Follow-up (tracer) studies, usually conducted as part of the process for evaluating training 
courses and programs, are also important sources of information for vocational training 
planners. Although more oriented to workforce supply than demand issues, follow-up studies 
can be used to monitor labor market trends and determine the geographic mobility, wage, and 
career patterns of former trainees. 

A follow-up questionnaire is developed, pilot-tested, and then mailed to course/program 
completers (graduates). (Guidelines for conducting follow-up studies are presented in Part 3 of 
this module — Supplemental Instruments.) Among other things, graduates are asked whether 
they (a) are employed full- or part-time, and (b) found a job for which they were trained. If 
graduates are employed in a different job, the studies also seek to find out how useful they 
consider their training to have been for their actual job. The questionnaire also asks graduates 
(a) about their wages, (b) how they found their job, (c) how long they looked before finding it, 
and (d) if different training would have made finding employment easier. (A prototype follow- 
up questionnaire is presented in Part 3 of this module — Supplemental Instruments.) An 
analysis of responses to these and other questions helps to determine the demand relevance of 
particular training courses or programs and sheds some light on the circumstances of entry into 
the labor market. 

For best results, follow-up study questionnaires ought to be short and simple. It is 
recommended that follow-up studies be conducted approximately 6 months after 
course/program completion, at the end of the first year, and again at the end of the second and 
third years. Important information revealed by a three-year follow-up includes answers to the 
following questions: 
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1. Is it getting harder or easier to find a particular job? 

2. Are graduates employed in jobs other than the one for which they were trained? 

3. Are there shifts in the location in which graduates are employed? 

4. What are the trends in earnings, by job title? 

It is not necessary, at the outset, to conduct large-scale studies in order to gain an initial 
idea of the situation. To begin with, a study can be made of small populations, giving priority 
to courses or programs of questionable efficacy and which create the most urgent problems 
(Bertrand, 1992, chap. 7). 

A significant problem is that unemployment rates, waiting periods, the acceptance of 
unrelated jobs in other geographic areas, and wages earned do not indicate on their own the 
success or failure of a particular course or program. This information needs to be 
supplemented by comparisons with (a) the employment record of a control group, (b) feedback 
from employer interviews, (c) different courses or programs, (d) other training institutions, and 
(e) local/regional employment conditions as well as trends. 

Advantages. Follow-up studies supplement other sources of labor market information by 
revealing what has actually happened to graduates in the labor market. They are not an 
instrument for long-term projections or forecasts, but they are a tool for monitoring training 
courses and programs and facilitating adjustments to meet labor market requirements. 
Information collected on the employment and wages of graduates provides effective signals on 
the balance of demand by job title in local and regional markets. 

Since the population surveyed is a group of former trainees who graduated at the same 
time and who were looking for employment at the same time and under similar conditions, 
vocational training planners can make good use of this tool for both labor supply and, to a 
lesser extent, demand and trend analysis. Follow-up studies are relatively inexpensive and can 
provide valuable information on knowledge and skill requirements. Furthermore, because they 
are being sent to graduates, a higher response rate can be anticipated (Campbell, 1996, chap. 
1 ). 

Disadvantages. Problems arise when too many objectives are set for follow-up studies 
other than those related to assessing how graduates have fared in the labor market. In 
addition, when questionnaires get unduly long or complicated, requiring considerable time to 
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complete, response rates and the quality of the answers diminish. Low response rates also 
result from out-dated or incomplete addresses. Furthermore, information collected much more 
than 6 months after graduation suffers from the problem of remembering specific details. As a 
result, questionnaires may be incomplete or include unusable responses. There is also a 
problem with the data collected — self-reported information (provided by graduates only) has 
questionable validity. 

In general, follow-up studies can contribute little to identifying the need for establishing 
new training courses or programs. However, this in no way diminishes their importance to 
vocational training planners and policy-makers when considering the merits of maintaining, 
expanding, improving, reducing, or discontinuing existing offerings. 



Note. The advantages and disadvantages of the five labor market signalling 
approaches can be easily compared by consulting Table 2 on the following 
pages. In addition, Table 3 lists the labor market signalling approaches and 
identifies the value of the information they provide. 
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Table 2. Advantages and Disadvantages of Labor Market Signalling Approaches 



Advantages 


Disadvantages 


Reporting by public 

• Services are in touch with local, regional, 
and state labor market happenings, collecting, 
compiling, analyzing, and reporting 
information on unemployment and present 

as well as future job openings 

• Planners can use available data in 
determining training needs 

• No sizable investment of new resources is 
required for generating and disseminating the 
data 


employment services 

• Data available is more or less incomplete 

• Data may not be representative of overall 
labor market situation 

• Information available is inadequate and 
could be misleading due to extent to which 
employers report job openings, lower level 
qualification requirements of jobs reported, 
and duplication of information with that 
from newspaper help-wanted ads 

• Services are more highly concentrated in 
urban areas catering to the formal sector, 
where they function mainly as a job exchange 
for the unemployed 

• Generation and dissemination of labor 
market information is performed as a side- 
line 
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Table 2 (continued) 



Advantages 



Disadvantages 



Analysis of job advertisements 



Provides current demand information and, 
over time, reveals employment trends 

Major newspapers contain voluminous 
sections of job advertisements 



Employer must be contacted for further 
details when information is imprecise and/or 
incomplete 

Focuses principally on metropolitan areas 



Analysis of advertisements complements 
signals received other sources 

Analysis could be undertaken by vocational 
training planners who could also discuss ads 
with the employers who floated them 

An easy and inexpensive approach to 
identifying jobs with favorable employment 
prospects 



Often includes more white- than blue-collar 
jobs and may be limited to certain sectors of 
the economy 

Variety and number of job openings 
underrepresented due to some employers 
not using want ads 

Some advertised jobs have already been 
filled or otherwise do not currently exist 

Number of ads in local or regional 
newspapers may be insufficient to construct 
a sample for analysis 



• Deals with past and present, not future, 
needs and does not predict future industry 
growth or change 
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Table 2 (continued) 



Advantages 


Disadvantages 


Key informa 

• Straight-forward and modest in cost 

• Information yields current indicators of 
workforce supply and demand imbalances in 
local/regional markets 

• Information gives early warning signals about 
significant changes in workforce supply and 
demand, and/or confirms trends previously 
documented 

• Information can be gathered on employers’ 
screening, hiring, and training practices 

• Can be conducted with individuals and 
groups, in the public and private sectors, as 
well as in formal and informal labor market 
sectors 

• Conduct of interviews and analysis of 
information can be entrusted to government, 
research institutes, universities, etc. 


nt interviews 

• Information is mainly qualitative, reflecting 
personal views and perceptions 

• Information is not representative in terms of 
research rigor 
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Table 2 (continued) 



Advantages 


Disadvantages 


Employe 

• Goes right to the source of labor market 
demand 

• Provides useful information in a reasonably 
short time 

• Employers’ willingness to cooperate is 
increased, because their views and 
suggestions are considered 

• Personal interview method provides a high 
response rate and yields precise and complete 
information on a host of relevant questions 


r surveys 

• Not all employers are able (or willing) to 
provide detailed predictions of occupational 
demand 

• Estimates provided by employers may be 
biased toward overestimation if they feel that 
their replies will positively affect the 
availability of skilled workers 

• Different employers may classify jobs in 
different ways, thus affecting the 
comparability of the information collected 

• Comparability suffers as changes in the 
workplace alter job requirements, especially 
when the jobs are not reclassified 

• Low level of confidence in the accuracy of 
survey information 

• Surveyed employers may not provide valid or 
reliable information, especially when asked 
for long-range estimates 

• Employer surveys do not cover skilled worker 
demand in the informal sector, particularly in 
rural areas 
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Table 2 (continued) 



Advantages 



Disadvantages 



Follow-up studies 



• Reveal what actually happened to graduates 
in the labor market 

• Monitor training courses/programs, thereby 
facilitating adjustments to meet labor market 
requirements 

• Information collected on employment and 
wages of graduates provides signals on the 
balance of supply and demand by job title in 
local/regional markets 

• Relatively inexpensive and produce useful 
results quickly 



• Problems arise when including objectives that 
are not related to how graduates have fared 
in the labor market 

• Response rates and the quality of answers 
diminish, if questionnaires are too long or 
complicated 

• Poor response rates result from out-dated or 
incomplete addresses 

• Information provided by graduates has 
questionable validity 

• Contribute little toward identifying the need 
for new training courses/programs 
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Table 3. Labor Market Signalling Approaches and 
the Value of the Information They Provide 



'^\Categories 

Approaches 3 


Workforce 


Knowledge 

anH clnll _ 


Employer practices 


Supply 


Demand 


aliU oAJll 

requirements'* 


Screening 


Hiring 


Public employment 


MV 


MV 


MV 


LV 


LV 


services 












Job advertisements 


None 


MV 


MV 


LV 


LV 


Key informant 


LV 


MV 


MV 


HV 


HV 


interviews 












Employer surveys 


None 


MV 


HV 


HV 


HV 


Follow-up studies 


HV 


MV 


MV 


None 


None 



Key. HV = High-value information LV = Low-value information 

MV = Moderate-value information None = No direct information 



Notes. 



“Each approach listed in column 1 was judged against the maximum potential value of the 
information which could be collected regarding five categories (headings of columns 2 - 6). This 
table shows that the choice of "best" approach depends on the information needed in addition to 
the particular advantages and disadvantages of each approach (as detailed in Table 2). 

The most reliable information about current and future workforce demand results from the 
assembling of data from two or more of the approaches listed in column 1. 

b A job and task analysis must be conducted to identify all the knowledge and skill 
requirements of a successful job incumbent. 
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Progress Check - Part 2 — Labor Market Signalling Approaches 

Directions. Read each of the following items carefully and write in your answer. If you have 
difficulty with any of the items, go back and review the relevant content before continuing. Check 
your answers against those provided in the Progress Check Feedback at the end of this module. 

1. Name the five labor market signalling approaches that were described. 

A. 

B. 

C. 

D. 

E. 

2. True or False: Public employment services determine and report the present and short-term 
future demand for skilled workers. 

A. True B. False 

3. True or False: The analysis of job advertisements in newspapers, etc., when supplemented by 
direct contact with the establishments/employers, provides quantitative information on future 
labor market needs. 

A. True B. False 

4. True or False: Workforce demand information from the analysis of job advertisements is 
inconclusive. 

A. True B. False 
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Progress Check - Part 2 (continued) 

5. List four essential preconditions for the satisfactory outcome of a key informant interview. 

A. 

B. 

C. 

D. 

6. Identify the most significant disadvantage of a key informant interview. 

7. Name two widely used methods of collecting information from employers. 

A. 

B. 

8. Explain why some employers are not able or willing to provide detailed workforce demand 

information. 

9. Name the labor market signalling approach used to determine workforce demand that is also 

highly useful for supply analysis. 

10. True or False: High rates of graduate unemployment, the acceptance of unrelated jobs, and 
low wages indicate the failure of a training program. 

A. True B. False 
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Part 3 — Supplemental Instruments 



A Comment 

The foregoing overview of the major workforce projection and forecasting approaches (Part 
1), and labor market signalling approaches (Part 2), is not exhaustive. Since the approaches 
described in Part 1 of this module will continue to be primarily the responsibility of economists 
or workforce planners/analysts, in view of their relevance to wider issues of employment and 
labor market policies, in addition to vocational training planning, those concerned with the 
latter might be satisfied with gaining overall knowledge in this respect. 

However, such knowledge must include an understanding of the advantages and 
disadvantages of each approach and its potential for determining labor market demand. It 
should enable vocational training planners to identify the approaches that they could undertake 
without having to wait for economists or manpower planners/analysts to fill labor market 
information gaps, which in some localities and regions are considerable. Reporting by public 
employment services, analysis of job advertisements, key informant interviews, employer surveys, 
and follow-up studies are strong candidates in this respect. It is for this reason that this last 
part of the module — Supplemental Instruments — presents (a) an example of the organization 
and analysis of key informant interviews, (b) guidelines for conducting follow-up studies, and (c) 
a prototype follow-up questionnaire. 

Owing to the fact that none of the approaches can claim predominant significance, the use 
of several approaches is preferable to the application of any one. Moreover, given the past 
emphasis placed on mechanistic workforce projection and forecasting approaches — with 
disappointing results — there is room to conduct a variety of labor market signalling 
approaches. 

The process nature of workforce demand and supply interactions, as reflected in [ever- 
changing] labor supply and demand imbalances, and the special importance of capturing these 
changes in a timely fashion also suggests the importance of labor market signalling approaches. 
These approaches involve routine data collection and analysis — rather than the mechanistic 
type workforce projections and forecasts which are made at a single point in time (Campbell, 
1996, chap. 1). 
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An Example of the Organization and Analysis of Key Informant Interviews 

The following example describes an individual in-person key informant interview process. 

It was performed as part of an assessment of the workforce and training requirements of a 
Forestry Development Strategy. The interviews were carried out by a labor market analyst and 
a vocational training planner. Their primary purpose was to supplement existing projections, 
which were thought to be too aggregate and contradictory, and not sufficiently up-to-date to 
serve as adequate planning inputs. 

The interviews were planned and organized in the following manner: The first step 
consisted of obtaining the names and contact information for experts in the forestry sector who 
were knowledgeable about workforce supply and demand. This was done through general 
discussions with individuals in forestry departments and institutions, as well as forest and timber 
businesses, during which the question was posed: Whom do you consider most knowledgeable 
on workforce demand and training in the forestry sector? Altogether, 15 experts were 
identified as potential key informants. However, only ten were located in the same geographic 
area, and judged to be qualified to serve as key informants. These ten key informants also 
provided a balanced representation of the different major forestry subsectors, both public and 
private. 

Four core questions were prepared. They addressed current and future workforce demand 
and supply imbalances, the main factors underlying such imbalances, and workforce issues likely 
to arise over the next 5 years. Five years was the time horizon visualized by the Forestry 
Development Strategy. A final question solicited further comments, suggestions, or elaborations 
on the four core questions. 

The key informants were asked the following questions: 

1. What occupational/skills category(s) — professional, technical, vocational — is currently 
in short supply? In oversupply? 

2. What occupational/skills category(s) is likely to face increases of demand in the next 5 
years? Likely to face decreases in demand? 

3. What factors will have an important influence on increases (or decreases) in demand? 

4. What will be the most critical workforce issues in forestry and timber industries over 
the next 5 years? 
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5. Would you please comment on the above questions, for example, how you plan to 
meet workforce shortages. 

Of the ten key informants, eight answered the five questions during a personal interview, 
for a response rate of 80%. The two others were very busy and could not schedule a personal 
interview, but did respond to some of the questions over the telephone. 

An analysis of the replies led to the following conclusions: 

1. The eight key informants agreed that qualified managerial and supervisory personnel 
were in the most critical shortage, followed by skilled loggers, skidder and other logging 
machinery operators. Two key informants expressed the view that forest rangers, technicians, 
and unskilled workers were currently in oversupply. 

2. There was general agreement that experienced managers, logging supervisors, and 
skilled loggers (competent in improved felling techniques) were badly needed in the immediate 
future. One key informant noted that he foresaw diminishing requirements for technical and 
vocational skill groups. 

3. With the increased export orientation of timber production envisioned by the Forestry 
Development Strategy, most key informants foresaw a need for an all-out effort of skill 
upgrading "across the board," that is, "a call for more highly skilled workers at all levels, from 
managers down to machine operators." In view of the urgent need for "harvesting and 
transporting the forestry crop economically" and meeting foreign competition, "by far, the most 
critical workers will be related to occupational groups engaged in these forestry operations." 

4. Most key informants referred to the brain-drain problem which went beyond the issue 
of skill development. An incentive system needed to be adopted for managers and senior 
skilled personnel, to stem the tide of "the incredible competition in the region. Otherwise, the 
country will end up training a high-level workforce for more developed countries . . . where 
most of our highly skilled workforce has migrated to." 

On the timber industry side, some key informants stressed the particularly urgent need for 
imparting higher-level technical training to operators such as sawers, machinists, saw doctors, 
and maintenance and repair workers for sawmilling and woodworking machines. The absence 
of publicly-supported training courses and programs for the timber industry was criticized. 
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This brief analysis of key informants’ responses largely confirmed the broad picture of the 
general workforce and skill needs, as put forward by existing projections. However, the 
interviews made it possible to pinpoint more explicitly major priorities of the training effort 
required to cope with current and prospective workforce supply and demand as well as skill 
requirements. 

Most of the key informants’ responses were admittedly of a qualitative nature. On the 
other hand, some key informants did provide — without having been specifically asked to do so 
— quantitative needs estimates. This pertained in particular to the specific subsector for which 
the informant concerned had primary responsibility. 

Another important conclusion drawn from these interviews is that (a) most key informants 
were willing to participate and cooperate during the interview (some of them stating it had been 
the first time they were asked about workforce demand and training needs pertaining to the 
labor force under their responsibility); and (b) carefully selected key informants are an 
important source of workforce information which, untapped, deprives workforce analysis and 
training needs assessment of a valuable contribution by those who are knowledgeable on 
workforce and training questions in the forestry sector. Moreover, the costs involved in 
obtaining this valuable information were modest. 
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Process Check - Part 3A — Supplemental Instruments 

Directions. Read each of the following items carefully and write in your answer. If you have 
difficulty with any of the items, go back and review the relevant content before continuing. Check 
your answers against those provided in the Progress Check Feedback at the end of this module. 

1. Describe the primary purpose for conducting the key informant interviews in forestry. 



2. Identify the first step in organizing the survey. 



3. List the agencies contacted to identify potential key informants. 

A. 

B. 

C. 

4. What was the survey response rate? 

5. The survey revealed that a shortage of 

and : personnel was the most critical. 

6. Most of the information reported in the analysis was ? . 

A. Quantitative B. Qualitative 

7. Identify two important conclusions about key informants drawn from the forestry sector 
survey. 

A. 



B. 




155 



135 



34 



Guidelines for 
Introduction 



Prepare for 
the study 



0 




Conducting Follow-up Studies 



The practical side of conducting a follow-up (tracer) study using a mailed 
questionnaire (survey instrument) to collect data is briefly presented here as 
an 11-step process. Mailed questionnaires are normally a cost-effective 
method of collecting quantifiable information from geographically dispersed 
as well as mobile individuals. 



1. Determine the resources such as time and expertise necessary for 
developing, pilot-testing, printing/duplicating, and mailing a 
questionnaire, as well as processing, interpreting, and reporting the data 
collected. Prepare a budget that shows estimated costs. 

2. Inform trainees who are about to complete their course or program 
about the purpose and procedures of the follow-up study. It is essential 
that trainees be encouraged to participate; make it clear that the purpose 
is to determine what happens to them after they graduate, so that future 
training can be more relevant to the labor market. 

3. Ask trainees to complete a baseline information form which includes 
their name, address, phone number, gender, age, work experience, etc. 
Add individual training performance data (attendance, grades, etc.) and 
statements about the trainee’s behavior to the form. This information is 
useful when interpreting data collected from the study. 

4. Set up a control group of individuals who will enter the labor market at 
about the same time as the trainees, but who did not follow a training 
course or program. This is an optional, but highly desirable step. The 
characteristics of these individuals should be similar to those of the 
trainees. A control group makes it possible to compare the labor market 
experiences of those who received training with the experiences of those 
who did not receive training. 

5. Develop the questionnaire. It is important to ask only those questions 
that will yield essential information. Demographic and other information 
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available from the baseline information form (step 3) or existing records 
is omitted and all nice- or interesting-to-know, but non-essential, items 
are left out. A prototype follow-up questionnaire follows these 
guidelines. It groups common questions and presents them in four parts. 
At the end of part one, which is completed by everyone, respondents are 
directed to proceed to either part two or part three depending on their 
employment status. As was the case with part one, all respondents are 
asked to complete part four. This questionnaire includes an appropriate 
combination of forced-choice items (e.g., Part 2, items 7 and 8) and open- 
ended items (e.g., Part 2, items 9 and 10). After deciding on the type(s) 
of items to use, they must be carefully worded so that (a) the response is 
not implied, and (b) all respondents interpret the questions as intended. 

6. Select a sample of trainees to pilot-test the questionnaire. During the 
pilot test, the trainees are asked to indicate if any items are unclear or 
difficult to answer. Revisions are made, if necessary, on the basis of the 
pilot-test results. Afterward, a high-quality questionnaire is printed in 
sufficient quantity. It may be desirable to color code the various parts or 
versions (e.g., second mailing, etc.) of the questionnaire for ease in 
sorting and use. 

7. Prepare a cover letter. It should be clear and to the point, providing the 
necessary explanatory information. An effective cover letter (a) states 
why the graduate has received the questionnaire, (b) identifies the 
purpose of the follow-up study, (c) explains why the questionnaire should 
be completed and returned, (d) assures confidentiality of the responses, 
(e) indicates approximately how much time it will take to complete the 
questionnaire, (f) provides directions on how and when to respond, and 
(g) expresses appreciation for participating in the study. This 
information may also appear on the printed questionnaire. The cover 
letter ought to be on letterhead stationery and each one should have an 
original signature by some official known by the graduates. 
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Conduct the 8. Mail the cover letter and questionnaire, along with a self-addressed, 
study stamped envelope, to all course/program graduates, or to a random 

sample if the population is large. When conditions allow, a telephone 
call, just prior to the mailing, will personalize the request and increase 
the rate of return. Send a second mailing to non-respondents after a 2- 
week waiting period. A follow-up telephone call to the remaining non- 
respondents may be necessary after an additional 2-week waiting period. 

9. Tally the data and compile the information from open-ended items and 
the comments provided. Hand-tally sheets can be used for small 
populations or when automated (computer) data summaries are not 
available. When tally sheets are prepared at the same time as the 
questionnaire, they show how the collected information will appear. 

This facilitates changes to items in order to simplify the tabulation and 
analysis of data and written responses. 



Prepare a 10. Prepare a report which documents follow-up study activities and the 

report results. The data can be presented using a variety of methods. For 

example, it can be outlined in narrative text and illustrated using tables 
and charts. Based upon an analysis and interpretation of the data and 
written responses, draw conclusions and make recommendations about 
the courses and/or programs studied. 

11. Collect information about the costs incurred in developing and delivering 
the training. This information is necessary to calculate the resources 
expended on the training of each participant, in terms of facilities, 
equipment, materials, personnel, and so forth. Such information 
facilitates cost-benefit calculations. This is an optional, but highly 
desirable, step. 



Information on planning and conducting a follow-up study is available in Module 
A- 10, Conduct a Student Follow-Up Study (American Association for Vocational 
Instructional Materials, 1978, pp. 3-62), which also provides further references. 
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A Prototype Follow-up Questionnaire 

Name Date 

Current address 

Home phone number ( 1 

The purpose of this follow-up survey is to find out about your employment since graduation. 

Your responses will be confidential. Please take a few moments today to complete this survey and 
return it in the enclosed, self-addressed, stamped envelope. Your responses are valuable and your 
assistance is greatly appreciated. Thank you. 

Parti (for EVERYONE") 

1. Course/program completed: 

2. What is your present status? (Check all that apply.) 

Employed by a private enterprise (company) 

Employed by a government agency 

Self-employed 

Working in a family business 

In the military service 

Pursuing further education/training (please specify) 

Other (please specify) 

Unemployed 

(If currently unemployed, proceed to Part 3) 




CONTINUED ON NEXT PAGE 



139 



153 



38 



Part 2 ( ONLY for those who are currently employed^ 

3. Are you working: (Check all that apply.) 

Full-time 

Part-time 

Locally 

In the job you were trained for 

In a job related to your training 

In a job that does not use your training 

4. What is your job title? 

5. How did you find this job? (Check one.) 

Direct application to employer (walk-in) 

Training institution job placement service 

Public employment service 

Private employment agency 

Friend or acquaintance 

Parent(s) or other relative 

Help-wanted ads (flyer, newspaper, journal, or magazine) 

Other (please specify) 

6. How long did you look (wait) before finding this job? 

Less than 1 month 3 to 4 months 

1 to 2 months More than 4 months 



) 



CONTINUED ON NEXT PAGE 
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Part 2 ('continued') 

7. What was your starting hourly wage rate? (Check one.) 

Less than $6.00 $10.00 to $11.99 

$6.00 to $7.99 $12.00 to $13.99 

$8.00 to $9.99 $14.00 or more 

8. What is your present hourly wage rate? (Check one.) 

Less than $6.00 $10.00 to $11.99 

$6.00 to $7.99 $12.00 to $13.99 

$8.00 to $9.99 $14.00 or more 

9. What subjects/topics from your training were most helpful in your job? 



10. Which subjects/topics were least helpful? 



1 1 . Would you have been employed without training? 

Yes No 

Comment: 



CONTINUED ON NEXT PAGE 
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Part 2 (continued') 

12. Did your training include all that was necessary to perform your job? 
Yes No 



Comment: 



13. Did your training qualify you for a higher -level job than your present employment? 

Yes No 

Comment: 

14. Have you received any job training from your employer? 

Yes No 

Comment: 



CONTINUED ON NEXT PAGE 
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Part 3 (ONLY for those who are currently unemployed! 

15. Are you actively seeking employment? 

Yes No 

Comment: 

16. Do you feel adequately prepared for a job? 

Yes No 

Comment: 

17. Would a different training course or program have improved your chances for 
employment? 

Yes No 

Comment: 

18. How long have you been looking for employment? 

Less than 4 months 4 months or more 

Part 4 ffor EVERYONE! 

19. Please add any particular observations, comments, and/or suggestions about the training 
you received and about possible ways to improve it. (For example: Needed more 
practice in performing job skills.) 



You are now finished with the survey. Please return it in the addressed and 
stamped envelope provided. Thank you for taking the time to help us. 



Note . Adapted from Education and Training for Work: Planning Programs (pp. 41-43), by C. P. 
O ampbell, 1996, Lancaster, PA: Technomic. Copyright 1996 by Technomic Publishing Company. 

ERIC 

143 it- 3 



42 



Progress Check -- Part 3B — Supplemental Instruments 

Directions. Read items 1 and 2 carefully and write in your answers. If you have difficulty, go 
back and review the relevant content before continuing. Check your answers against those 
provided in the progress check feedback at the end of this module. 

1. List the seven steps in preparing for a follow-up study. 

A. 



B. 



C. 



D. 



E. 



F. 



G. 



2. Identify the advantage of using a control group when conducting a follow-up study. 



LEARNING ACTIVITIES 

1 . Prepare a draft questionnaire for use in a follow-up study. Compare it with the prototype 
questionnaire provided in this part of the module. 

2. Pilot-test the questionnaire on a sample of trainees and make all appropriate revisions. 
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Progress Check Feedback - Part 1 — Workforce Projection and Forecasting Approaches 

(pp. 9-10) 

1. List the three workforce projection and forecasting approaches described in Part 1. 

A. Manpower [ workforce ) requirements approach (pp. 4-5) 

B. Econometric models (pp. 5-6) 

C. Input-outvut models (dp. 6-7) 

2. Identify the approach that relates forecast economic growth to the output of VET 

institutions. Manpower [ workforce ) reguirements approach (p. 4) 

3. Name the reasons for the difficulty in accurately converting from sectoral/sub-sectoral to 
more specific occupational employment possibilities in the workforce requirements 
approach. 

A. Overly optimistic estimates (at the sectoral levels of employment growth (v. 5 ) 



B. Increasingly rapid economic change (p. 5) 



C. Lack of_allowance for technological advances (p. 5) 



D. Lack of allowance for changing business practices (d. 5) 



E. Lack of allowance fop occupational mobility (_ resignation s, layoff^ and discharges ) (p. 5} 



F. Lack of_ allowance for workers withdrawing from and returning to the workforce (e.g., 
quit to raise a child or pursue further education, then return ) (p. 5} 
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Progress Check Feedback -- Part 1 (continued) 

4. Identify two models that provide an alternative to the first step of the workforce requirements 
approach. 

A. Econometric models (p. 5) 

B. Input-output models (p. 6} 

5. Name the workforce projection and forecasting approach that takes into account inter- 
industry relationships. Input-output models (p. 6) 

6. Name the steps which characterize an input-output analysis. 

A. A projection is made of the output of a particular industry (p. 6} 

B. A workforce coefficient is applied to the absolute increase in production (p. 6} 

C. The pwjection and workforce coefficient are translated into VET requirements (p. 6) 
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Progress Check Feedback -- Part 2 — Labor Market Signalling Approaches 

(pp. 27-28) 

1. Name the five labor market signalling approaches that were described. 

A. Reporting by public employment services (pp. ii-ii) 

B. Analysis ofjob advertisements (pp. 11 & 13-14) 

C. Key infonnant interviews (pp. 11 & 14-16) 

D. Employer surveys (pp. 11 &. 16-18) 

E. Follow-up (_ tracer ) studies (pp. 11 & 18-20) 

2. True or False: Public employment services determine and report the present and short-term 

future demand for skilled workers. 

A. True B. False X 

(... the data is seldom analyzed and, few, Harry decisions are 
cached ... — p. 12: The full potential to do this is far from being 
adequately developed — p. 13} 

3. True or False: The analysis ofjob advertisements in newspapers, etc., when supplemented by 

direct contact with the establishments/employers, provides quantitative information on future 

labor market needs. 

A. True B. False X 

(... current labor market demand and ... — pp. 13-14) 

4. True or False: Workforce demand information from the analysis of job advertisements is 
inconclusive. 

A. True X B. False 

(„. and must be compared with data from other approaches — p. 14} 
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Progress Check Feedback — Part 2 (continued) 



5. List four essential preconditions for the satisfactory outcome of a key informant interview. 

A. Careful selection of experts from (a) local /regional businesses ; (b) public agencies ; 

(c) economic development authorities ; and (d) employe / ; worker^ and professional 

organizations (oo. 14-15) 

B. Use of_a structured interview schedule (p. 15} 

C. Timely analysis of_ the information obtained (p. 15} 

D. Regular and frequent follow-up of_the signals captured (p. 15} 

6. Identify the most significant disadvantage of a key informant interview. Provides mainly 

qualitative infonnation which reflects personal HgM (P- IQl 

7. Name two widely used methods of collecting information from employers. 

A. Mailed questionnaires (p. 16} 

B. Personal mterviews (p. 16) 

8. Explain why some employers are not able or willing to provide detailed workforce demand 
information. ... employers don’t gather the data necessary for reliable estimates (p. 17} 



9. Name the labor market signalling approach used to determine workforce demand that is also 

highly useful for supply analysis. Follow-up (tracer^ studies (dp. 18219} 

10. True or False: High rates of graduate unemployment, the acceptance of unrelated jobs, and 

low wages indicate the failure of a training program. 

A. True B. False X 

(This mfomiation needs to be supplemented by comparisons with 
(a} the employment record of_a control group* (hi feedback from 
employer interviews, (c) different courses or programs, (d) other 
training institutions, and (e) local/regionql employment conditions as 
weU as trends — p. 191 



Progress Check Feedback — Part 3A — Supplemental Instruments 

(p. 33) 

1. Describe the primary purpose for conducting the key informant interviews in forestry. 

To supplement existing projections, which were though to be too aggregate and contradictory 
and not sufficiently up-to-date (p. 30) 

2. Identify the first step in organizing the survey. Obtaining the names and contact information 

for averts in the forestry sector (v. 30} 

3. List the agencies contacted to identify potential key informants. 

A. Forestry, departments (p. 30} 

B. Institutions (p. 30} 

C. Forest and timber, businesses (p. 30) 

4. What was the survey response rate? 80% (v. 31) 

5. The survey revealed that a shortage of qualified managerial (v. 31 ) 

and supervisory (p. 31) personnel was the most critical. 

6. Most of the information reported in the analysis was ? . 

A. Quantitative B. Qualitative X 

(dp. 31-32) 

7. Identify two important conclusions about key informants drawn from the forestry sector 
survey. 

A. Key informants were willing to participate (p. 32} 

B. Carefully selected key informants are an important source of workforce 

information (p. 32} 
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Progress Check Feedback -- Part 3B — Supplemental Instruments 

(P-42) 



1. List the seven steps in preparing for a follow-up study. 
A. See pages. 34-35, steps 1-7 



B. 



C. 



D. 



E. 



F. 



G. 



2. Identify the advantage of using a control group when conducting a follow-up study. 

Makes it possible to compote labor market experiences of_ those who receive training with the 
experiences ofjhose who do not (p. 34, step 4} 
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Human Resource Development Department 



The University of Tennessee 



SATISFACTION RATING INSTRUMENT 



Directions: Please respond to each of the following 17 statements by circling the number that corresponds to 
your level of disagreement or agreement. The scale is 1 = strongly disagree to 9 = strongly agree. The 
instructional material phrase (below in bold type) is the preliminary part of each statement. 



The instructional material used (module) 
Statements 

1 . was satisfying to me. 

2. suited my learning style. 

3. was enhanced by the inclusion of 
learning objectives. 

4. increased my professional 
vocabulary. 

5. taught knowledge along with skills. 

6. maximized learning. 

7. was frustrating to me. 

8. encouraged me to learn. 

9. required excessive effort for the 
benefits received. 

10. embedded learning in the 
application environment. 

1 1. included illustrations, examples, and 
questions which facilitated learning. 

12. provided continuous support 
throughout the learning process. 

13. enabled me to acquire relevant 
experience. 

14. limited my ability to learn. 

15. provided training on specifics. 

16. included all that was necessary to 
perform. 

17. enhanced my motivation and 
facilitated learning. 



Strongly disagree 

1 2 3 



1 



4 

4 

4 



4 

4 

4 

4 

4 



4 

4 

4 



5 

5 

5 

5 

5 



6 

6 

6 



6 

6 

6 

6 

6 



6 

6 

6 



Strongly agree 
7 8 9 



8 

8 

8 

8 

8 

8 

8 



9 

9 



9 

9 

9 

9 

9 



9 

9 

9 



Please jot down what you liked about the instructional material. 



Please jot down what you didn’t like about the instructional material. 



Thank you. Your feedback will remain confidential. 
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Human Resource Development Department 501 
Survey of Human Resource Development 



The University of Tennessee 
Assignment Sheet 



COURSE DELIVERABLE - LABOR MARKET DEMAND 



Course Objective : Consider the merits of various labor market signaling approaches for 

collecting information on the supply/demand balance of skilled workers. 

Subject Matter Topic : The salient characteristics of labor market signaling approaches 

References : 

1 . Textbook, Education and Training for Work: Planning Programs (Chapter 1) 

2. Handout (module), Determining the Demand for Skilled Workers 
Assignment : (Responsibilities and requirements) 

1 . Identify and describe concisely the salient characteristics of any two (2) of the five labor 
market signaling approaches discussed in the references. Include a table showing their 
advantages and disadvantages. 

2. Staple pages in top left comer and submit without cover stock or plastic page protectors. 
Due Date : 

1. Milestone submittal date . 



Human Resource Development Department 501 
Survey of Human Resource Development 



The University of Tennessee 
Progress test No. 5 



DETERMINING TRAINING NEEDS 



Directions: Each of the following items contains four (4) alternatives, one of which provides the 



best answer. Circle the letter which precedes the best answer. 



1. Training and development needs result from 

A. current human performance not being what is 
required by organizational objectives. 

B. future responsibilities requiring new skills. 

C. organizational goals of helping employees achieve 
their potential. 

D. all of the above. 

2. A customer survey may be carried out when investigating 

training needs because 

A. performance problems may exist that are unknown 
within the organization but that frustrate customers. 

B. customers are more aware of the overall effectiveness 
of operations than those within the organization. 

C. most training and development needs can be 
identified in this way. 

D. customer complaints are important. 

3. A good policy when conducting needs surveys is to 

A. concentrate only on the target group in order to 
increase objectivity. 

B. carefully avoid placing constraints on how the survey 
results will be used before the survey is carried out. 

C. concentrate on concrete responses and avoid eliciting 
attitudes, motives, and other subjective responses. 

D. survey three levels: the target group, their 
supervisors, and their subordinates. 

4. Organizational audits reveal training needs by 

A. using surveys of perceived training needs. 

B. exposing performance deficiencies that show up in 
records of the results of operations. 

C. eliciting opinions concerning future training needs 
from a panel of informed managers within the 
organization. 

D. establishing a system of assessment methods that do 
not rely on existing records. 
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The University of Tennessee 
Information Sheet 



NEEDS ASSESSMENT: A RATIONALE AND METHODOLOGY 



Introduction 

Simply stated, needs assessment is the process of identifying and clarifying human 
performance problems. As defined by Lampe (1986), it is "an effort to reveal the gaps between 
what people do at work and what their employers would like them to do" (p. 101). Needs 
assessment is a process that provides the information necessary for informed decision making. 
Kaufman (1987) maintains that "a needs assessment provides the direction for useful problem 
resolution through identifying, documenting, and selecting appropriate problems for training" 

(p. 20). 

Before a trainer considers planning or providing a training course or program, a properly 
conducted needs assessment should have determined that the identified problem is one which can 
be solved through training. A needs assessment which shows that a problem cannot be solved by 
training alone is just as important as one which shows that training is the answer. It is the 

t 

decision maker’s ethical and professional responsibility to analyze all alternative interventions and 
make recommendations accordingly. 

The conduct of a needs assessment should be planned, organized, systematic, and 
goal-directed. Abel (1986) recommends that "Goals should be specific, measurable, and 
objective" (p. 70), and categorized according to either a functional or individual level. On the 
functional level, they should relate directly to the accomplishment of the organization’s goals. 

On the individual level, a distinction should be made between what a worker needs to know and 
what he or she should be able to do (p. 70). 
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Training Limitations and Alternative Interventions 

Notwithstanding all its value — or perhaps on that very account, tra inin g should not be 
proposed as the answer to those organizational problems that are better approached in other ways. 
After a problem has been identified, it ought to be thoroughly investigated to determine its cause 
before training is prescribed as a bona fide solution. For example, training is not the answer 
when: 

1. The knowledge, skills, and attitudes to be trained cannot be used on the job. 

2. A change in behavior cannot be observed or measured. 

3. The job environment will not permit the performance of the behaviors acquired. 

4. Workers are capable of performing job tasks to standards, but for a variety of reasons, 
are unmotivated or unwilling to do so. 

5. The organization’s goals can be accomplished more effectively through other 
interventions. 

6. There are insufficient financial, human, physical, or technological resources to complete 
training. 

If any of these situations exist, training will be an ineffective response to the organizational 
problem. A different intervention or possible training as part of a broader plan of action should 
be considered. 

When the knowledge and skills to be trained cannot be used on the job, it is usually because 
of an incorrect decision during the planning function. The mistake can be the result of 
inadequate or faulty information at the time of the original plan. It can also come from changing 
conditions on the job between the planning and plan implementation. Before any training is 
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undertaken, the skilled worker needs of the immediate future must be evaluated. And they must 
be evaluated carefully! 

If the change in worker behavior cannot be measured, then it should not be trained. Unless it 
is possible to get a specific measure of trainee performance before, during, and after training, the 
employer will never know whether the effort failed or succeeded. Trainers must answer the 
question, "What will have to happen to show me that the trainee has attained job task standards?" 

Measuring change becomes a problem when learning objectives are (a) not used, 

(b) incomplete or inconsistent with job task performance, or (c) phrased in ambiguous language. 
For example, a poorly-written objective like "The trainee will understand the importance of 
maintainin g accurate records of the patient’s temperature" is abstract and impossible to observe 
and measure. Properly stated, the learning objective would read: "Measure a patient’s 
temperature using an oral thermometer to within one-half of one degree and record the reading 
on the patient’s chart in accordance with clinical procedures and without assistance." Now the 
trainee, trainer, and all who work with the learning objective can determine when proficiency has 
been attained. 

When workers are capable of performing job tasks to standards, but are unmotivated or 
unwilling to do so, training will not solve the problem. There may be many causes for this 
unwillingness — the job may be monotonous, boring, or workers may lack the incentive to do it 
well. The absence of feedback may cause workers to doubt the value of their performance. On- 
the-job conditions may be such that the worker is not rewarded for performing tasks properly. 
The worker may not even know the job task standards. Group pressures and group norms may 
be causing reduced production or lower standards. Those who perform job tasks properly may 
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be penalized. In none of these cases would training the worker in how to do the job better bring 
about any appreciable improvement in productivity or standards. 

The. question to ask in this situation is "If the workers’ jobs depended on it, could they 
perform the job tasks properly?" Training is appropriate only if the answer is no. If a worker is 
capable of performing the job tasks to standards, but is not doing so, other interventions are 
indicated. The following eight side-headings provide information on non-training interventions. 
Communicate Expectations 

Before workers can perform effectively, they must know exactly what they are expected to do 
and what acceptable performance looks like. By identifying the job tasks and providing 
performance standards, then adding positive consequences for proper performance, workers will 
tend to repeat their good performance. 

Add Feedback 

When workers don’t know how well they are doing, that is, when they don’t see the results 
of their accomplishments or receive feedback concerning their performance, they will most likely 
never improve. Workers need to know when they are meeting expectations and when they are 
not. Feedback in itself motivates workers; it gives them more reason to care, to perform well. 
Periodic feedback can produce dramatic results, and it is a relatively inexpensive intervention. 
Eliminate Adverse Physical and Environmental Conditions 

Undesirable environmental and physical conditions in the workplace hamper the effective use 
of knowledge and skills. Bad equipment, materials, and supplies, as well as conditions which 
cause health and safety problems, all inhibit job performance. 

Workers are often prevented from implementing trained behaviors by attitudinal or social 
obstacles. Whenever newly acquired skills and abilities conflict with closely held attitudes and 
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values, the work environment will not support their use. For example, training Saudi Arabians 
to be waiters and cooks will be fruitless if they must work in a region of the country where social 
mores mitigate against Saudis being employed in these jobs. In such cases, the attitudinal 
environment must be changed — either before training starts, or as part of a broader 
developmental strategy. 

Similarly, to train workers in ways that maximize quality and productivity will be futile if the 
work group tends to penalize those who exceed group norms. In this case, the on-the-job 
environmental obstacle will interfere with or prevent implementation of the trained behavior. 
Before tr ainin g, those who are responsible should assure themselves not only that it is possible to 
develop the desired behaviors — but also that group pressures at the workplace can be modified 
to accept the new knowledge, skills, and attitudes. 

Enrich the Job 

The trend toward simplification and specialization has resulted in many "impoverished" jobs. 
Some are so narrow, monotonous, and tiresome that workers do not maintain satisfactory levels 
of performance. Apart from adding variety through job rotation, jobs can be redesigned to 
encompass a complete and meaningful piece of work (product). This increases interest and leads 
to a sense of accomplishment. Enriched jobs add value and importance which motivate workers. 
Pav Attention to Interpersonal. Group, and Intergroup Behaviors 

Workers within an organization may be more motivated toward their individual or small 
group goals than toward the organizational goals. Organization development (OD) could be the 
solution when the organization needs fixing — not the workers’ behavior. The process reveals 
how individual and small group goals and those of the organization can and should overlap, 
resulting in a more effective organization. 
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Recognize Performance 

How a reward and incentive system is structured affects job performance positively or 
negatively. If workers are to be motivated to put forth maximum effort, to do their best, 
reasonable wages must be provided and recognition, as well as other rewards, must be linked to 
performance. 

The general level of wages received by workers and the differences in pay among them can 
lead to problems. Wage and salary administration ought to establish a justifiable rank order of 
different jobs and set wages that are logical and equitable. The content of jobs and the 
requirements necessary for proficient performance, not custom, favoritism, or an imprecise 
notion of the job’s value in the workplace should determine the differential rates of pay. 

Recognition and tangible rewards for accomplishments nurture and emotionally drive workers 
to expend additional energy. They contribute significantly to the overall well-being of any 
workplace. 

Opportunities to get ahead in the organization are also important. There needs to be a 
rational systematic promotion plan that recognizes and rewards proficient performance and 
demonstrated potential. 

Add Positive Discipline 

Problems of human conduct or social relationships generally make themselves quite clear. 
Habitual lateness and absenteeism are problems that cannot be ignored. Lack of effort, laziness, 
and dishonesty are others. Patterns of belligerency; lack of respect for management, 
organizational rules, or policies; and trouble getting along with co-workers or customers also 
indicate that a reprimand or tactful discipline may be in order. When discipline is handled with 
impartiality and good judgment, the worker’s performance will likely improve. 
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Only a small percentage of workers cause disciplinary problems. Furthermore, the reasons 
for such behavior may have nothing to do with the job. Family problems, chemical dependency, 
emotional difficulties, and financial or legal difficulties are often the root cause. Problem 
workers are expensive to have on the payroll and difficult to supervise. They tend to upset the 
morale of the work group. If a supervisor wants a worker to be more cooperative and 
productive, every effort should be made to deal with the underlying reasons for the unsatisfactory 
behavior. 

Positive discipline begins with counseling, which includes (a) listening patiently to what the 
worker has to say, (b) refraining from criticizing or offering hasty advice, and (c) trying to 
determine if the worker is troubled by something deeper than what appears on the surface. 
Counseling interviews are aimed at helping workers unburden themselves. The payoff comes 
when the worker gains confidence in the supervisor and consequently doesn’t vent resistance and 
frustration on the job (Bittel, 1974, pp. 286 & 404). 

Use On-the-Job Coaching 

Coaching involves on-the-job development where a supervisor supports an individual’s 
progress through a planned set of learning experiences. According to research by the General 
Electric Company, 90% of employee development is done this way. While this may not be true 
in all organizations, it does emphasize that coaching is an important method for inspiring workers 
to improve their knowledge, skills, and attitudes. 

Supervisors must identify specific things that workers can do to improve quality and 
productivity, then make a commitment to help in their accomplishment. They must take 
advantage of opportunities to guide workers to their full potential in an environment that 
motivates them to excel. 
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Summing Up 

While examining situations where training is not appropriate, eight alternative interventions 
were presented. All things considered, training is an appropriate solution to job-related problems 
where workers have actual or potential deficiencies of knowledge or skill (Laird, 1985, p. 83). 
This can happen when (a) new workers are hired, (b) a worker is promoted or transferred, 

(c) new jobs are created, (d) existing jobs are to be performed in new ways, (e) new technology 
or equipment is introduced, or (f) there is a need to improve safety. However, before a decision 
is made to undertake training, planners ought to consider five more alternatives. 

1. Change employee selection methods. This can increase the pool of skilled workers. 

2. Improve job performance methods and use existing knowledge and skills more efficiently. 

3. Assign workers to jobs that fully use their expertise. 

4. Involve workers in planning, problem-solving, and decision-making that affects them. 

5. Introduce new technology. 

6. Reorganize and adjust priorities or change business practices. 

The alternatives discussed and listed may be more efficient and/or cost effective than training 
would be. Consequently, determining the appropriateness of training is of paramount importance 
in building credibility with an organization. 

Needs Assessment Methods 

The best source(s) of accurate and complete information, whether documents or people, ought 
to be identified. This is true even if the source proves to be difficult to access or use because of 
expense, unavailability, or unwillingness. 
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Document Review 

Workplace documents which can provide clues to problems as well as other pertinent 
information include: (a) employee records such as performance appraisals, absenteeism and 
turnover data, complaints and grievances; (b) position (job) descriptions; (c) strategic business 
plans and budget reports; (d) production and quality control records; (e) accident and critical 
incident reports; (f) regulatory requirements; and (g) exit interviews. Data can be collected from 
these and other documents without interrupting workers and asking them for information which is 
already available in workplace records and reports. 

A check with the personnel department will reveal which employee records are accessible. 
Since privacy laws and regulations have become stringent, some of these records may be 
considered privileged information (Donaldson & Scannell, 1986, p. 24). 

There are many methods and combinations of methods for collecting information from human 
sources. They run the full gamut, from an informal interview to a complex formal research 
design. Some of the best methods are: 

1 . Interviews (one-on-one and group) 

2. Observations on the job 

3. Survey techniques (questionnaires, checklists, rating scales, etc.) 

4. Performance and achievement tests 

5. Work samples 

6. Advisory committees 
Interviews 

There are two main types of interview formats. The one-on-one, where the interviewer 
meets with an individual worker in person or talks with him over the telephone, is the more 




BEST COPY AVAILABLE 



183 



163 



10 



common type. Less common, but also effective, is a group interview, where the interviewer 
meets with several workers collectively. The group interview tends to elicit information that 
individuals might not bring up on their own. However, it may stifle honest discussion. Either of 
these formats can be structured, using prepared questions, or highly unstructured and informal, 
taking place over a cup of coffee or during lunch. If the conversation is initiated by the person 
being interviewed, so much the better. If not, tactful questioning should be used. Interviews can 
be quite effective at revealing causes of and possible solutions to problems (Donaldson & 
Scannell, 1986, p. 23). 

A structured interview is more formal, generally occurring during a pre-scheduled meeting. 
Previously prepared questions usually solicit information concerning: 

1 . Problems encountered on the job 

2. Improvements that could be made 

3. Most- and least- liked parts of the job 

4. Accomplishments on the job 

5. Personal goals 

6. Felt needs for acquiring additional knowledge or skills 

It is extremely important that the questions and interview itself not be perceived as prying or 
playing a clandestine role. In addition, it is always necessary to respect the confidentiality of the 
responses. The advantages and disadvantages of the one-on-one (both in person and over the 
telephone) and group interview formats are presented in Appendix A. 
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Observations 

Training needs can also be discovered with little or no interference with operational activities 
through on-the-job observations of worker performance. The observation can be as formal as a 
time and motion study or as behaviorally specific as watching a new worker operate a machine. 

It may be as unstructured as walking through an office and hearing overtones of conflict or 
misunderstandings between workers. 

A complaint from a co-worker about telephone messages, outgoing correspondence, etc. may 
indicate a need for secretarial training. Observed problems such as poor housekeeping, hazards, 
time- wasting practices, and so forth may be traced to an ineffective supervisor or a need for 
training. Observations can be used to distinguish between effective and ineffective interpersonal, 
group, and intergroup behaviors; however, the presence of an observer may bias the behavior 
witnessed. Consequently, observations should be checked and verified by repetition or by 
comparison with those of other competent observers. The advantages and disadvantages of the 
observation method are listed in Appendix A. 

Survey Techniques 

Survey techniques range from simple response sheets to sophisticated questionnaires. Surveys 
are used to reach a large number of people, or people located over a large geographic area, in a 
relatively short period of time. A commonly used approach is to prepare a list of questions about 
training needs, and then solicit opinions from workers, their supervisors, and other stakeholders. 
The training needs are then identified by analyzing the data collected. 

Since the resources (time and money) required to survey an entire population are often 
unavailable, a randomly selected or stratified sample may be surveyed instead. A variety of 
question formats may be used, including the open-ended, forced-choice, and priority-ranking 
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types. Unfortunately, know-how, time, and hard work are required to (a) develop the questions, 
(b) write explicit instructions, (c) format and pilot-test the questionnaire, and (d) compile the 
information collected. See Appendix A for a list of the advantages and disadvantages associated 
with survey techniques. 

Performance and Achievement Tests 

Job task performance can be assessed through criterion-referenced performance tests. Pre- 
employment and periodic performance testing will reveal deficiencies in knowledge or skills, 
thereby establishing a need for training. 

A comprehensive achievement test facilitates the process of "testing out". This makes it 
possible to eliminate repetitive training in knowledge and skills already attained. Additionally, it 
reveals the magnitude of the training yet to be accomplished. 

Work Samples 

Tangible samples of work such as management reports, or less tangible examples like how a 
manager conducts a meeting, can identify problem areas that require further analysis. This 
unobtrusive method provides direct information on the organization’s actual work. It requires an 
analyst who is familiar with the work process and product standards. 

Advisory Committee 

A training advisory committee, composed of leaders representing key departments and 
committees/groups within an organization, can serve as a "sounding board" for new program 
ideas and act on training requests. Meeting agendas should be developed, distributed in advance, 
and followed. Decisions and recommendations should be recorded and brought to the attention of 
management. When involved in a positive and meaningful way, committee members will be the 
best advocates for training (Donaldson & Scannell, 1986, p. 25). Even though "has-beens" and 
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semi-retired individuals can frequently be spared to attend meetings, they should not be included 
for obvious reasons. 

Other Sources of Information 

1. Behavior assessment. Instruments that measure the work behavior of individuals and 
groups include tests of attitudes, self-reports, opinionnaires, simulations, games, and other such 
devices. 

2. Consultations. Discussions with colleagues in the same or a similar industry can also be 
helpful. 

3. Requests from management. Because management support is needed for tra inin g 
courses and programs, these requests should be investigated, but not overreacted to. However, 
top-level management is the major information source regarding organizational strategy, current 
policies, and future expansion or down-sizing. Consequently, they may know what training will 
be required to fulfill these special needs. 

Choosing a Method 

Although all possible sources of accurate and complete information should be consulted to 
ensure a successful needs assessment, "criteria will vary for different organizations in different 
situations" (Blackburn, 1987, p. 48). Time schedules, resources, and costs must be considered in 
choosing which needs assessment method to use (Lampe, 1986; Kaufman, 1987). However, 
Lawrie (1986) maintains "data closest to a situation not only is most likely to be valid, but also 
has the best chance of being accepted" (p. 22) by both the selected workers and the organization. 

The results of a study which surveyed human resource departments in America’s top 
companies revealed that (a) informal discussions (interviews) was the most frequendy used needs 
assessment method, and (b) observations was the next most frequent. One may have expected to 
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find the more scientific needs assessment methods rated higher than the unstructured and informal 
interview. Nonetheless, even when a more formal method is used, experienced training 
professionals realize that "informal discussion is often needed to gain the necessary commitment 
from the clients to accomplish a project or program" (Ralphs & Stephan, 1986, p. 74). 

Training needs can be either perceived (felt) needs or demonstrated (measurable) needs 
(Bowman, 1987; Lampe, 1986). Whichever the case, involving the intended trainees in 
determining their training needs "may have the largest impact on design decisions" (Lampe, 

1986, p. 101). It is also important to solicit and listen to workers’ ideas in order to "build [their] 
commitment to the training program" (Bowman, 1987, p. 32). When participating in the needs 
assessment process, workers should be assured that their responses are confidential and will be 
taken seriously by management (Abel, 1986). 

After collecting information from the potential recipients of training, additional data can be 
collected from their co-workers, supervisors, and subordinates to get a comprehensive profile of 
training needs. When discrepancies are discovered among these different populations, the trainer 
should, if necessary, investigate further. Differing opinions sometimes indicate a communication 
problem within the organization, which provides another potential training need (Bowman, 1987). 

Data should also be collected from top management, because they can provide pertinent 
information regarding the company’s long-term goals and strategies. Furthermore, the success of 
a needs assessment often depends on the support of management. Research clearly indicates that 
the more management participates in the needs assessment, the greater the likelihood of their 
support (Abel, 1986; Bowman, 1987; Swanson, 1987). 

Customers or clients are another source of valuable information. They often perceive 
problems in areas that insiders are not aware of (Lampe, 1986; Ralphs & Stephan, 1986). 
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Although training requests should certainly be acted upon, needs assessments ought to be 
conducted in a proactive mode as well as in a reactive mode. Assessing training needs should be 
an ongoing process, with the trainer always being on the alert to potential problems that can be 
prevented through training (Kaufman, 1987). Bowman (1987) asserts "programs [should] flow 
from the assessment, not assessment from suggested programs" (p. 30). 

In instances where a needs assessment indicates more than one training need, the needs 
should be prioritized. Kaufman (1987) suggests assigning monetary values to each need "in 
terms of what it will cost to reduce or eliminate the need and what it will cost to ignore the need" 
(p. 82). 

Preparing a Report 

Once the training needs have been analyzed and prioritized, a report should be prepared and 
presented to management for approval and support. The report should contain (a) the purpose 
and objectives of the needs assessment, (b) methodology used, (c) data sources, (d) the analysis 
of information collected, (e) results, and (f) a recommendation as to the appropriate plan of 
action. Appendix B shows a worksheet that is useful in preparing a needs assessment report. 

In Conclusion 

The success of training begins with conducting a needs assessment. A properly conducted 
assessment actually saves time and money. It enables the organization to make informed 
decisions and to realize a return on the investment of resources. A needs assessment may 
provide erroneous information, however, if it is conducted in a "hit-or-miss" fashion. The result 
would be an inadequate and disappointing training course or program that reflected poorly on 
training as a profession. 
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A needs assessment should be properly conducted in a systematic manner to maintain 
training’s professionalism and integrity and to gain respect from management as a profitable 
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activity. Galagan (1986) asserts, "As training struggles to establish itself as a credible 
contributor to bottom-line results, it must be able to conduct, report, and justify effective needs 
assessment" (p. 4). 

Tr ainin g can’t solve all problems — not all problems are training problems. 



ERIC 



170 



190 



17 



References 

Abel, L. (1986, November). "The value of training needs analysis." Magazine of Rank 
Administration , pp. 70-72. 

Bittel, L.R. (1974). What every supervisor should know (3rd, ed.). New York: McGraw-Hill. 

Blackburn, M. (1987). "Needs assessment: Where’s the secret formula?" In New Directions 

for Training and Development: Problems and Solutions . Ames, Iowa: American Society 
for Training and Development, Iowa Chapter. (ERIC Document Reproduction Service 
No. ED 289 976) 

Bowman, B. (1987). "Assessing your needs assessment." Training . 24(1), 30-34. 

Donaldson, L. & Scannell, E.E. (1986). Human resource development, the new trainer’s guide 
(2nd. ed.). Reading, MA: Addison-Wesley. 

Galagan, P. (1986). "What the world needs now." Training and Development Journal . 40(8), 4. 

Kaufman, R. (1987). "A needs assessment primer." Training and Development Journal . 41(10), 
78-83. 

Laird, D. (1985). Approaches to training development (2nd. ed.). Reading, MA: Addison- 
Wesley. 

Lampe, S. (1986). "Getting the most out of needs assessments." Training . 23 (19). 69-76. 

Lawrie, J. (1986). "Revitalizing the HRD function." Personnel . 63 (6). 20-25. 

Leach, J. (1979). "Organization needs analysis: A new methodology." Training and 
Development Journal . 33(9). 66-69. 

Ralphs, L.T. & Stephan, E. (1986). "HRD in the Fortune 500." Training and Development 
Journal . 40(10), 69-76. 




131 

171 



18 

Swanson, R.A. (1987). "Training technology system: A method for identifying and solving 
training problems in industry and business." Journal of Industrial Teacher Education . 
24(4), 7-17. 





172 



Page 1 of 3 



APPENDIX A 

Advantages and Disadvantages of Needs Assessment Methods 



Advantages 



Disadvantages 



One-on-one (in-person) Interview 



• High response rate. 

• Individuals are generally more willing to 
talk than to fill out a questionnaire. 

• Misunderstood questions can be clarified. 

• Appropriate for jobs involving 
considerable personal judgment or mental 
application (intellectual tasks). 

• Yields precise, complete, and comparable 
information. 

• Additional information revealed through 
non-verbal messages (body language). 

• Appropriate with illiterates and those with 
language difficulties. 

• Responses can be recorded on audio tape 
or through written notes. 



• Expensive and time-consuming to prepare 
for and conduct a number of individual 
interviews and compile the information 
collected. 

• Results depend upon the cooperation of 
those interviewed. 

• Can interfere with operational activities. 

• Requires a skillful, understandable 
interviewer with a pleasant personality and 
the ability to listen. 

• Individuals tend to say they do what they 
think they should do, not what they 
actually do. 

• Interviewer’s bias, demeanor, and 
appearance can distort responses. 

• Requires a private, distraction-free setting. 

• Note-taking may be disturbing and audio- 
taping threatening to some individuals. 

• Scheduling interviews with busy 
individuals may be difficult. 

• Individuals may give conflicting 
information which must be reconciled. 



One-on-one (telephone) Interview 



• High response rate. 

• Individuals are generally more willing to 
talk than to fill out a questionnaire. 

• Misunderstood questions can be clarified. 

• Appropriate for jobs involving 
considerable personal judgment or mental 
application (intellectual tasks). 

• Comparatively inexpensive. 

• Information collected on the job or at 
home in a short period. 
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• Less sensitive than in-person interviews. 

• Requires a trained, understandable 
interviewer with a positive attitude and 
good telephone manner. 

• Individuals tend to say they do what they 
think they should do, not what they 
actually do. 

• Reaching busy individuals to be 
interviewed or having them return a call 
may be difficult. 

• As the use of telemarketing increases, 
some individuals will hang up, not taking 
the time to distinguish between an 
interview and a solicitation. 
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Advantages Disadvantages 

Group Interview 



• Few people involved. 

• Misunderstandings can be clarified. 

• Participants generally find the activity to 
be a stimulating and rewarding 
experience. 

• Appropriate for jobs involving 
considerable personal judgment or mental 
application (intellectual tasks). 

• Information collected and verified in a 
short time. 

• Consensus can be reached on conflicting 
terminology or other points of 
disagreement. 

• Information provided by one individual 
can serve as a stimulus to others for 
recalling additional information. 



• Full-time commitment, requires time away 
from the job, can be expensive. 

• Requires a recorder and a trained, 
understandable interviewer (facilitator) 
well- versed in group dynamics. 

• Requires knowledgeable individuals who 
are able and willing to communicate and 
cooperate as a group. 

• Presence of supervisors may affect job 
incumbents’ participation. 

• Requires a distraction-free and 
comfortable setting. 



On-the-job Observation 



• Accomplished directly by observer or 
indirectly with a motion picture camera or 
an audio and video tape recorder. 
Simultaneous recording of personal 
observations is recommended. 

• Finds out what incumbents actually do, 
not what they say they do. 

• Few individuals involved. 

• Little if any interference with operational 
activities. 

• Appropriate for jobs with observable 
psychomotor tasks, especially when an 
incumbent, even with assistance, cannot 
describe what is actually done and how it 
is done. 

• Best way to obtain an understanding of 
job context or environment. 



• Time and safety factors may preclude 
direct observation. 

• Not able to observe all possible 
conditions. 

• Expensive and time-consuming to observe. 

• Presence of observer or recording device 
may cause stress and/or bias performance. 

• Ineffective for jobs involving considerable 
personal judgment or mental application. 

• Observations should be checked and 
verified, whenever possible, by repetition, 
or by comparison with those of other 
competent observers. 

• Requires a trained and knowledgeable 
observer who is patient and 
nonthreatening. 

• Methods oriented and slanted to the 
incumbent’s way of doing things. 
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Advantages 



Disadvantages 



Survey Techniques 



• Useful when a large number of individuals 
perform the same job. 

• Appropriate for jobs involving 
considerable personal judgment or mental 
application (intellectual tasks). 

• Can survey a large number of 
geographically distant as well as mobile 
individuals economically by mail. 

• Yields large amounts of quantifiable 
information in a relatively short period. 

• Questionnaire can be completed, on the 
job or at home, at the individual’s 
convenience. 

• Bias is eliminated on questions that could 
be sensitive or embarrassing when asked 
by an interviewer. 

• All individuals are asked the same 
questions in the same way. 

• Responses can be stored, manipulated, 
analyzed, and reported by a computer. 

• No need for trained observers or 
interviewers. 



• Requires know-how, substantial time, and 
hard work to (a) develop questions, 

(b) write explicit instructions, (c) format 
and pilot the instrument, and (d) compile 
the information collected. 

• A comprehensive questionnaire may be 
unduly long. 

• Places a heavy demand on recall by those 
completing it. 

• Questions may be misinterpreted. 

• Some individuals may be distrustful and 
decline to complete the questionnaire or 
not provide honest, thoughtful responses. 

• May be returned incomplete or filled in by 
unintended or inappropriate individual. 

• Low response rate, especially when 
mailed and there is no monetary incentive 
or persistent follow-up. 

• Address may be incomplete or inaccurate 
and questionnaire may not be forwarded. 

• Does not facilitate further probing. 

• May require a computer to manipulate the 
large amount of information collected. 

• Inappropriate for low literacy groups. 
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APPENDIX B 



TRAINING NEEDS ANALYSIS WORKSHEET 



Requester: Date- 


1 Task requiring improvement: . 






2 Frequency of task performance: 






3. Consequences of improperly performed task: 








4 Reason(s) task requires improvement: 






S Training rprommpnr1ation{<;)* 






6. Training action plan, if applicable (identify individual/department responsible for delivery 

of training and the delivery method recommended)- 










(Training Analyst) (Date) 

Reviewed By: 

(Training Manager) (Date) 

(Requester) (Date) 

(Supervisor) (Date) 

□ Approve Recommendation □ Reject Recommendation 

Jf recommendation is rejected identify an alternative solution- 







Human Resource Development Department 501 
Survey of Human Resource Development 



The University of Tennessee 
Progress test No. 6 



THE PERFORMANCE AUDIT 



Directions: Each of the following items contains four (4) alternatives, one of which provides the 

best answer. Circle the letter which precedes the best answer. 



NOTES 

1 . Why do organizations conduct performance audits? 

A. To increase the relevance and helpfulness of training to 
employees. 

B. To distinguish between problems that are solvable 
through training and those that are not. 

C. To find solutions for performance problems. 

D. All of the above. 

2. A goal of the economic aspects of a performance audit is to 

A. decide whether economic benefits justify efforts to 
improve specific performances. 

B. reduce the extent to which training is based on economic 
factors. 

C. provide accurate estimates of training costs. 

D. substitute human behavior measures for economic 
measures. 

3. When a manufacturer had a product quality problem, 
studies showed that machine operators were not able to 
make needed adjustments nor were they aware of quality- 
control results. Using the performance model, managers 
would probably conclude that 

A. a training program would be sufficient to solve the 
problem. 

B. employees needed training, quality-control results, and 
knowledge of concern about quality. 

C. quality standards would have to be changed. 

D. the problem could be solved most easily by replacing 
the workers. 

4. Improving employee job performance is most likely to have 
a beneficial result when 

A. the best performer is currently much better than the 
worst performer. 

B. the best performer and the worst performer are 
currently nearly equal in performance. 

C. the contributions of different types and levels of 
employees cannot be determined. 

D. both the best and worst performers are very good. 
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Human Resource Development Department 501 
Survey of Human Resource Development 



The University of Tennessee 
Progress test No. 7 



WORKFORCE PLANNING 



Directions: Each of the following items contains four (4) alternatives, one of which provides the 



best answer. Circle the letter which precedes the best answer. 



1 i An organization's plans for the development and use of human 

resources should 

A. increase productivity and profitability by concentrating on 
needs for specific skills and abilities. 

B. develop goals which combine meeting the organization's 
needs with efforts to promote individual growth. 

C. define individual growth goals only in terms of 
organizational needs. 

D. avoid the loss of efficiency resulting from attempts to 
interrelate individual and organizational goals. 

2. The most important requirements for bringing about planned and 

systematic organizational change are 

A. the recognition of felt needs and the support of influential 
people. 

B. budgets and personnel exclusively devoted to change. 

C. numerous personal conflicts within the organization to 
justify change. 

D. emotional equality and fairness currently existing in the 
organization. 

3. The Johari window, when used in career planning, rests on the 

assumption that 

A. organizational goals and individual goals are ultimately 
incompatible. 

B. communication of information between the organization and 
individuals is the least important aspect of career planning. 

C. manpower planning is a more important function than career 
planning. 

D. open two-way communication enhances interpersonal 
relationships. 

4. Job posting 

A. is usually desired by management but resisted by employees. 

B. is an effective way to give employees control of their career 
planning. 

C. gives the organization full control over who will succeed to 
each position. 

D. is not recommended in organizations with formal career 
planning programs. 
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Human Resource Development Department 501 
Survey of Human Resource Development 



The University of Tennessee 
Progress test No. 8 



INSTRUCTIONAL SYSTEMS 



Directions: Each of the following items contains four (4) alternatives, one of which provides the 



best answer. Circle the letter which precedes the best answer. 



1 . The most effective training programs are based on 

A. job-relevant behaviors. 

B. general principles of work effectiveness. 

C. behaviors that are not specific to a particular job. 

D. a content analysis. 

2. In the early analysis of a proposed instructional program, 
the most critical step is 

A. establishing the body of information trainees will be 
required to learn. 

B. developing a task list of the behaviors desired. 

C. deciding on the methods and techniques to be used by 
instructors. 

D. assessing the training potential of proposed trainees. 

3. When it has been decided to measure the trainees' level 
before they enter a training program, the goal should be 
to 

A. obtain an objective measure of potential ability. 

B. sample the entire range of aptitudes that apply to the 
trainees' positions. 

C. measure the extent to which trainees already possess 
the knowledge and skills to be trained. 

D. develop a measure that will accurately predict the 
instructional criteria. 

4. A behavioral objective is a 



A. description of the overall goals of a training system. 

B. test used at the beginning and end of instruction to 
measure improvement. 

C. description of the desired behavioral action, 
performance conditions, and standards of attainment. 

D. description of the behaviors required of instructors to 
put the instructional system into practice. 
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Instructional Systems Development 
for Accountability 



Clifton P. Campbell 



II est primordial d’assurer que le contenu et les r6sultats 
d’un cours de formation sont pertinents et utiles 6 l’6tu- 
diant. Aux Etats-Unis, on a mSme intents des proems con- 
tre des institutions qui, all6gue-t-on. n’ont pas r6ussi 6 
preparer retudiant ad6quatement pour le monde du tra- 
vail.Une approche qui peut assurer la responsibility est le 
ISD (en fran$ais: Elaboration des systbrnes d’instruction)/; , 
Les cinq etapes du systbme sont d6crites: analyse, design, 
elaboration, mise en oeuvre et contrdle. A la fin, -il est 
observe que ce processus est couteux et complique, et; 
q u’il importe d’en observer soigneusement toutes les 
etapes s’il doit reatiser tout son potentiel. : 7 : ’ • *: : 



F rom time to time the news 
media carry disparaging stories 
which imply that vocational education 
and training is ineffective. Some well- 
intentioned educators also produce 
data to back up their arguments that 
certain programs are not effective. 
Unfortunately, we even hear of pro- 
grams being criticized by the very stu- 
dents and employers they serve. The 
inference is that the full potential of 
vocational education and training to 
adequately prepare people for work is 
not being realized. 

The public is in a mood to hold 
administrators and instructors ac- 
countable for instructional program 
results — or the lack of them. Mal- 
practice suits have been filed, in the 
United States, claiming that schools 
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Dean of Instruction of Delaware Technical and 
Community College and had previously been a 
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Delaware and University of Maryland. Dr. Camp- 
bell has also taught at the secondary school level 
and worked as a surveyor. 



failed to prepare students adequately 
for the working world. One conse- 
quence of this mood is that many con- 
cerned people are insisting upon struc- 
ture and substance in instructional 
programs, 1 so that they are designed 
and carried out in a way that assures 
every student a base of skills, a body of 
knowledge, and unswerving attention 
to quality in teaching. 

Years of debate by leaders in voca- 
tional education and training have not 
yet produced anything like a consen- 
sus on the best course of action for 
bringing about desirable changes. But 
many experts see cause for optimism 
in an approach, borrowed from 
government and industry, which is 
aimed at making instructional pro- 
grams accountable for their results. 
This approach is called Instructional 
Systems Development (ISD). By defi- 
nition, ISD is a deliberate and orderly 
process for planning, developing, 



•The term instruction can be read as education 
and/or training. 



implementing and evaluating instruc- 
tional programs which ensures that 
students are taught the knowledge, 
skills and attitudes essential for suc- 
cessful job performance. 

The accountability of instructional 
programs is virtually certain when the 
principles and processes of ISD are 
used. This is because the derivative 
and iterative character of the method- 
ology ensures, when its procedures are 
faithfully carried out, that instruction 
meets job requirements. 

Parts of this methodology have been 
used by vocational educators and 
trainers for some time, But, the ISD 
process itself has only recently been 
refined into detailed procedures which 
are useful to curriculum developers, 
instructors, administrators and teach- 
er trainers. This article will describe 
these procedures and provide the 
reader with the framework for a quick 
understanding of ISD. It simplifies 
and compresses into a digest what is 
included in each phase of the ISD 
model. 



Canadian Vocational Journal, Vol. 18, No. 4 

ilST COPY AVAILABLE iso 2G0 . 




INSTRUCTIONAL SYSTEMS DEVELOPMENT MODEL AND FLOWCHART 

FIGURE 1 



Background 

The ISD approach for the develop- 
ment and accomplishment of voca- 
tional education and training pro- 
grams provides a methodology for 
gathering and analyzing job informa- 
tion, developing instructional mate- 
rials and media, conducting instruc- 
tion and evaluating and improving the 
effectiveness of instructional pro- 
grams. This approach is based upon 
evidence that each task/ behavior per- 
formed on a job can be identified and 
analyzed and that appropriate learn- 
ing objectives and instructional stra- 
tegies to achieve these objectives can 
be developed and structured so that 
instruction will be efficiently and effec- 
tively conducted. The application of 
the systems approach to instructional 
development ensures that all inter- 
related factors are considered and that 
the students, when the instruction is 
appropriately delivered, will reach 
performance standards required for 
specific job entry level proficiency. 

Model for Instructional 
Systems Development 

Instructional Systems Development 
consists of five major phases: I) Analy- 
sis, II) Design, III) Development, IV) 
Implementation, and V) Control. 
Each is a separate and distinct func- 
tion. The ISD procedural model and 
flowchart, exhibited in Figure 1, 
shows these phases, each of which is 
further divided into a number of 
blocks or steps. Symbolically, this 
structure outlines the recommended 
relationship of the nineteen procedu- 
ral steps. It also provides a graphic 
portrayal of the interrelationships 
between all the phases and blocks 
which form a closed loop instructional 
system. Components of the model are 
necessarily depicted in a linear 
sequence. In practice, some of the 
steps may be conducted concurrently 
while others must be done sequen- 
tially. 

Phase I, Analyze 

Before designing and developing 
instruction it is necessary to establish 
what constitutes adequate job perfor- 
mance. Job analysis is the basic 
method for identifying and describing 
the tasks and elements necessary to 
perform a clearly defined, specific job. 
It can involve any or all of the tech- 
niques of questionnaire surveys, jury 
of experts, personnel interviews and 
observations in the work environment, 
and reviewingjob task lists from occu- 



pational surveys. Regardless of how 
well all the other procedural steps are 
carried out, if job analysis data are not 
valid and reliable the resulting instruc- 
tional program will fail to produce 
graduates who can perform success- 
fully on the job. 

Some tasks are seldom required on 
the job and only minimum job degra- 
dation would result if they were not 
performed. However, other tasks may 
be highly critical to successful job per- 
formance, and the complex nature of 
the task makes instruction essential. 
Economic, time and other considera- 
tions require a decision as to which 
tasks are selected for instruction and 
which are not. Selection criteria and 
information collected during the job 
analysis are used in making this deci- 
sion. 

Job performances measures are 
constructed for tasks selected for 
instruction. In addition to evaluating a 
job incumbent’s task proficiency, job 
performace measures are used in writ- 
ing learning objectives (block 11,1), in 
preparing test items (block II-2) and in 
measuring the performance of gradu- 
ates on the job (block V.2). Because of 
their multiple uses, job performance 
measures are fundamental to the 
development and control of instruc- 
tion. However, while worthwhile, they 
are costly to develop. 

In order to avoid unnecessary dupli- 



cation of effort, existing course docu- 
mentation and other available mate- 
rial should be examined. This effort 
may reveal that all or portions of the 
analysis and other phases have already 
been done by someone else. 

As a final step in phase I, the tasks 
selected for instruction are analyzed in 
order to identify the most appropriate 
instructional setting(s). e.g., classroom 
and/or shop instruction, on-the-job 
training, etc. The optimal setting is the 
one that provides the necessary 
equipment, facilities and pesonnel as 
well as the most effective and efficient 
instruction. It should be noted, how- 
ever, that settings are usually selected 
by higher authority and trade-offs are 
frequently made based on resource 
availability and other constraints. 

Figure 2 shows the sequence of 
events required in the analysis phase of 
course development. Event outputs 
are listed alongside in terms of 
products. 

Phase II, Design 

Beginning with this phase the model 
is concerned with designing instruc- 
tion using the products from Phase I. 
The first step is the conversion of each 
task selected for instruction into a 
terminal learning objective. Job per- 
formance measures provide the basis 
for developing these terminal learning 
objectives. 




Canadian Vocational Journal, Vol. 18, No. 4 



201 BEST COPY AVAILABLE 



EVENTS: 

- phase 



ANALYZE 



1.1 


ANALYZE 

JOB 


1 


1.2 


SELECT JOB 
TASKS/FUNCTIONS 


l 


1.3 


CONSTRUCT JOB 
PERFORMANCE MEASURES 


l 


1.4 


ANALYZE 

EXISTING COURSES 


1 


1.5 


SELECT 

INSTRUCTIONAL SETTING 




Learning objectives (both terminal 
and enabling) must realistically de- 
scribe the knowledge and skills that 
are to be mastered by the student. 
They are the common three part objec- 
tive that provides a statement of the 
behavioral actions, performance con- 
ditions and attainment standards 
which can be measured within the 
learning and testing environment. 

Each terminal learning objective is 
subjected to a hierarchical analysis to 
derive intermediate (enabling) objec- 
tives. These enabling objectives pro- 
vide the level of detail necessary to 
specify the behavioral elements and 
the skills and knowledge that mediate 
a terminal objective. 

Learning objectives bridge the gap 
between performing a task under job 
conditions and learning how to per- 
form that task in an instructional set- 
ting. Precisely stated learning objec- 
tives serve as a control over the content 
and output of instruction. They define 
for the student, instructor, curriculum 
developer and test writer exactly what 
is required of them. 

After all the learning objectives have 
been prepared, test items to measure 
student attainment are written. These 
test items are called criterion refer- 
enced measures because they test 
against the criteria for successful per- 



PRODUCTS: 

1.1 a list of job tasks, the equipment and or 
materials involved, the conditions and 
standards of performance, etc. 

1.2 a list of tasks selected for instruction 

1.3 a validated job performance measure for 
each task selected for instruction 

1.4 an examination of the job analysis, task 
selection and job performance measures 
for any existing instruction to determine 
if these courses are usable in whole or in 
part 

1.5 selection of the instructional setting for 
tasks selected for instruction 



formance that were identified in the 
objectives. With a criterion referenced 
measure, students are rated on their 
ability to achieve the objectives. How 
others score on a test has no bearing on 
an individual’s grade. Sufficient items 
should be written on all learning objec- 
tives to allow for alternative forms of a 
test in order to facilitate retesting. The 
grouping of test items into pre-tests, 
progress tests and a post-test is done in 
block II.4. 

Once the test items have been deve- 
loped, those that were designed for 
lower level enabling objectives should 
be administered to a representative 
sample of the incoming student popu- 
lation, in the form of an entry level 
test, to measure entry level behavior. 
Entry level behavior includes the 
skills, knowledge and attitudes stu- 
dents possess at the time they report 
for instruction. Estimates of student 
capabilities will be verified or adjusted 
according to test results. When the test 
shows that some of the enabling objec- 
tives upon which test times are based 
have already been mastered by the 
students, those enabling objectives 
and their corresponding test items 
should be deleted from the course. 
Whenever the test indicates that stu- 
dents lack the prerequisite knowledge 
and/or skills, additional learning 



objectives and test items must be deve- 
loped to fill the gap between entry level 
assumptions and the true entry level 
required for the course. No modifica- 
tions are necessary when the test accu- 
rately measures the students’ entry 
level behavior. 2 Entry level behavior 
may also be assessed when actual 
instruction begins. Changes in entry 
population capabilities can be deter- 
mined in this way. 

Finally, the refined list of learning 
objectives, both terminal and ena- 
bling, are sorted, sequenced and 
grouped. This is usually done on the 
basis ofcommonality of subject matter 
and anticipated transfer of learning. 
The sequence is determined primarily 
by the dependent relationships among 
training objectives. Independent and 
coordinate objectives are sequenced in 
accordance with such principles as 
simple to complex, familiar to unfa- 
miliar, and job order. 

Because test items mesure specific 
learning objectives, they are structured 
at the same time as the learning objec- 
tives. This greatly facilitates the group- 
ing of test items into pre-tests, progress 
tests and a post-test. 

The sequence of events in the design 
phase is illustrated in Figure 3. The 
products of each event are also listed. 

Phase III, Develop 
The instructional development 
phase begins by identifying the learn- 
ing activities which must take place in 
order to produce the desired learning 
outcomes. The purposes of block III. 1 
are to: 1) classify the learning objec- 
tives according to the appropriate 
learning categories and sub-categories 
(see Table 1), 2) identify for each sub- 
category those learning guidelines 
necessary for optimum learning to 
take place (see Table 1), and 3) specify 
activities that must take place in the 
learning environment to provide 
instruction directly related to task per- 
formance. 

The four learning guidelines listed in 
Table 1 seem to apply to most of the 
learning categories. However, each 
learning sub-category does not require 
all or the same learning guidelines for 

: Results for each test item will probably show 
that some students already know the material 
while others do not. It is unlikely that eve- 
ryone tested will either know all of the mate- 
rial or not know it. Therefore, in establishing 
an entry level for a group paced course, cur- 
riculum developers must use their best judg- 
ment in determining the exact level of instruc- 
tion at which to begin the course. 
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11.1 learning objectives, both terminal and 
enabling 

1 1.2 a pool of criterion referenced test items 
which measure each learning objective 

11. 3 test(s) to establish student entry behav- 
iors and, when necessary, revisions to 
learning objectives and test items based 
on true entry levels 

11. 4 a list of sequenced and structured learn- 
ing objectives 



FIGURE 3 
Table 1 

LEARNING CATEGORIES AND SUB-CATEGORIES 



Learning Categories 
I. Mental skill 



II. Information/knowledge 
III. Physical/manual skills 



IV. Attitude 



Learning Sub-Categories 

1. Rule learning and using 

2. Classifying-recognizing patterns 

3. Identifying symbols 

4. Detecting 

5. Making decisions 

6. Recalling bodies of knowledge 

7. Performing gross motor skills 

8. Steering and guiding-continuous 
movement 

9. Positioning movement and recal- 
ling procedures 

10. Voice communicating 

11. Attitude learning 



Source: NAVEDTRA 106A, Phase III Develop, p. 11 



LEARNING GUIDELINES 



1. Inform Student (s) of Learning Objectives. At the beginning of each lesson the 
instructor will state the terminal objective. The learning objectives will also be 
specified in written materials. 

2. Provide for Active Student Practice. Whenever possible, write learning activ- 
ities that call for active responses from the student(s). Have them do some- 
thing to show that they understand the subject matter. 

3. Provide Guidance and Prompts to Student. Give student(s) sufficient exam- 
ples of what to do and explanations of how to do it so that they know what is 
expected of them. 

4. Provide Student(s) with Feedback. Tests should be used as teaching tools by 
fairly testing attainment of the objectives and providingstudent(s) with either 
reinforcement for correct answers or an explanation of their mistakes. Regu- 
lar classwork should also be written to provide the student(s) with feedback 
on their performance. 

Source: NAVEDTRA 1 10, p. 105 



optimum learning to take place. The 
purpose here then is to select those 
guidelines which best enable the stu- 
dent to master the learning objective. 
Based on the learning guidelines 
selected, as well as the learning objec- 
tives and test items, it is possible to 
identify what the instructor or instruc- 
tional materials must do to teach a 
student to perform the learning objec- 
tives and also what the student must 
do to show mastery of the subject mat- 
ter. 

The procedures in block III. 2 
involve two principal steps. Step one is 
the specificaton of an instructional 
delivery system which provides the 
optimal combination of instructional 
methods, media and organizational 
system to accomplish the terminal 
objectives. However, since there is no 
single best method of teaching which 
applies to all learning situations or 
objectives, the instructional delivery 
system will normally include a combi- 
nation of teaching methods, selected 
instructional media and a system of 
organizing students and instructors, to 
accomplish an objective. The selection 
of one delivery system over another, 
therefore, is made on the basis of care- 
ful analysis of the instructional situa- 
tion fromseveral standpoints: 1) learn- 
ing objectives, 2) course content, 3) 
student population, 4) instructional 
staff, 5) space, 6) equipment and facil- 
ity constraints, 7) instructional mate- 
rials, 8) time and 9) costs. Conse- 
quently, the instructional delivery 
system must be compatible with the 
learning objectives selected in block 
II. 1, the learning activities selected in 
block III. 1 and any other significant 
inputs from previous blocks in the 
model. 

The second principal step is the 
preparation of an instructional man- 
agement plan which is a set of proce- 
dures used to assure a smooth flow of 
students through the instructional sys- 
tem. This plan indicates exactly how 
the course is to be conducted, how the 
students are to be managed, when and 
where they will be tested, what the 
instructors and other support person- 
nel are to do and how each of the many 
elements within the plan work 
together. 

Developing instructional materials 
is both costly and time consuming. 
Therefore, it is essential to search out 
and consider the usefulness of existing 
materials before committing resources 
to the development of new instruc- 
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tional materials. Candidate materials 
should be examined to determine 
whether they can be used or adapted 
for use. 

In block II. 1, learning objectives 
were derived for each task selected for 
instruction. In block III. 1 , specific 
learning guidelines and learning activi- 
ties were identified that require differ- 
ent instructional treatment. From 
block III. 2 the media and methods of 
instruction have been specified and 
from block III. 3 the acceptable off- 
the-shelf instructional materials have 
been assembled. At this point all the 
preceding analysis and planning will 
be utilized to produce the necessary 
additional instructional materials. 

The production process includes 
developing draft materials using the 
learning activities specified in block 
III. 1 . These materials are tried on stu- 
dents, revised as necessary, and finally 
sent to the appropriate production 
specialists for development. A variety 
of approaches are available for use. 
The appropriate mix of these 
approaches will depend in large part 
on the available time and the facilities 
and resources that can be used. Lec- 
tures, video tapes, slide/ tape presenta- 
tions, job performance aids, and for- 
mal on-the-job training are all devel- 
oped according to the same general 
principles. The prescribed procedures 
allow for ample internal content 
review of the materials to ensure that 
they are doctrinally correct. 

The heart of the development phase 
is the validation of instructional mate- 
rials until the students who use them as 
planned meet the learning objectives. 

If the instructional materials 
selected in block III. 3 and those deve- 
loped in block III. 4 have been pro- 
duced efficiently, they will have the 
minimum possible elaboration. The 
instruction should be “lean” to ensure 
the economies of minimal instruction. 
When this material is tried on students 
for the first time, it should reveal some 
short-comings. These inadequacies 
can be corrected through the process 
of revision. If, on the other hand, 
tryout reveals only a few errors and 
difficulties, and students seem to grasp 
everything quickly, it is likely that the 
instruction has been overdone. 

Selected members of the target pop- 
ulation usually go through the mate- 
rials individually at first and revisions 
are made on a basis of those trials. 
Then the number of students is 
increased in order to detect more pos- 
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sible errors. Finally, when the mate- 
rials are thought to be complete, they 
are tested on a sufficient number of 
students to demonstrate their effec- 
tiveness at an acceptable level of con- 
fidence. 

As the materials are improved, 
fewer students will encounter difficul- 
ties, and more students will work 
through them to an acceptable level of 
performance. At this point the instruc- 
tional materials are ready for quantity 
reproduction and implementation. 

Figure 4 illustrates the sequence of 
events in the development phase. 
Event products are listed alongside the 
chart. 

\ 

Phase IV, Implement 

The implementation of the instruc- 
tional management plan is the termi- 
nal step in planning and preparation 
and occurs just before instruction be- 
gins. It is at this point that any discre- 
pancies or deficiencies in the materials, 
procedures, tests and other necessary 
components of the total instructional 
program are identified and corrected. 

It is also necessary at this time to 
make sure that the instructors and 



ill. I the classification of learning objectives 
by learning category and sub-category as 
well as the determination of appropriate 
learning guidelines and activities 

111.2 an instructional delivery system and sup- 
porting media as well as the instructional 
management plan for conducting the 
instruction 

111. 3 selected existing instructional materials 
adequate for use and materials which 
must be revised before use 

Ul.4 the development of instructional mate- 
rials for all objectives where existing 
materials are not available 

111.5 validated instructional materials 



other key personel are adequately 
trained in the techniques or proce- 
dures with which they are not familiar. 
The emphasis is on assuring that the 
right people are at the right place at the 
right time, that they know what they 
are supposed to do, and that they have 
the materials, equipment, facilities and 
other resources necessary to do it. 

A complete instructional package 
containing learning objectives, tests, 
instructional materials/ instructional 
management and a trained staff 
should now be available. What 
remains to be done is to conduct 
instruction in accordance with the 
procedures and documentation con- 
tained in the instructional manage- 
ment plan. As part of the instructional 
activity, required changes and other 
observations are documented. Records 
are also kept on evaluative data and 
problems students have with instruc- 
tional materials. At the completion of 
each course cycle, the collected infor- 
mation should be used to improve 
instruction for the succeeding cycle. 

The sequence of events for the 
implementation phase is shown in 



FIGURE 4 




Canadian Vocational Journal, Vol. 18, No. 4 

184204 



BESTCOPY AVAILABLE 



EVENTS: 

PHASE 

IV 

IMPLEMENT 



IV. 1 IMPLEMENT INSTRUCTIONAL 
MANAGEMENT PLAN 

_ _ L ~ 

IV.2 CONDUCT 

INSTRUCTION 



Figure 5. Event outcomes are also 
listed in terms of products. 

Phase V, Control 

This phase provides for the evalua- 
tion and revision of instruction. The 
first activity, internal evaluation, is 
conducted to determine if the instruc- 
tion is providing students with the 
necessary knowledge and skills to meet 
the learning objectives developed in 
Block 11.1. This is accomplished by 
assessing student test performance and 
evaluating the effectiveness of instruc- 
tional materials, media and methods. 
The internal evaluation process con- 
sists of collecting pertinent perfor- 
mance, progress and process data and 
information from students, instructors 
and other informed personnel, eva- 
luating this data and formulating 
recommendations. 

External evaluation occurs when 
graduates are on the job. It uses job 
performance as the standard to evalu- 
ate proficiency and the adequacy of 
the instructional process. In a sense 
this revalidates the original task list. 
External feedback is usually obtained 
by following graduates into the field 
either physically or by questionnaire. 
The most reliable and costly external 
feedback is gained by administering 
the job performance measures pro- 
duced in block 1.3. As with internal 
evaluation, external evaluation is a 
continuous process which identifies 
problems and recommendations for 
improvements/ revisions to instruc- 
tion. 

The final function in the ISD pro- 
cess is that of maintaining quality con- 
trol of the instructional program 



PRODUCTS: 

IV. I documents containing information on 
time, space, student and instructional 
resources and stall trained to conduct the 
instruction 

IV. 2 a completed cycle of instruction with 
information needed to improve it for the 
succeeding cycle 



through the process of revision. Since 
a considerable amount of effort has 
gone into the design and development 
of an instructional program, the deci- 
sion about what to revise, if anything, 
should be made only after careful 
analysis of the inputs from blocks V. I 
and V.2. Generally, revision means 
those changes made in the procedures, 
techniques and materials in the pro- 
gram. The emphasis is usually on 
change of technique rather than of 
content, doctrine, or curriculum. 

Figure 6 shows the sequence of 
events for the evaluation and quality 
control phase. Event products are also 
listed. 

Commentary 

ISD is a methodology for making 
vocational education and training 
programs accountable for their 
results. This is possible because ISD 
ensures that instruction is relevant to 
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PHASE 
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the job. Its iterative and derivative 
character virtually assures that 
instruction will meet job requirements 
if procedures are conscientiously exe- 
cuted. However, ISD procedures are 
difficult, time consuming and costly to 
carry out. Consequently, ISD will be 
worth the cost and effort only to the 
extent that it is implemented so as to 
realize its potential. 

The extent to which users adhere to 
the ISD process and actually use its 
products in designing training is sub- 
ject to question. In practice, steps are 
sometimes omitted and the class con- 
nection between components, that is 
essential to make the process truly 
derivative, is not maintained. Most 
important, the testing and revision 
necessary to ensure job relevance gen- 
erally do not occur. When this 
happens the potential of ISD to ensure 
that instruction meets job require- 
ments is not being realized. Efforts to 
implement ISD should therefore con- 
centrate on maintaining the integrity 
of the model by guarding against 
omissions in the process and failure to 
use its products. 
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USING EXTERNAL RESOURCES 



Directions: Each of the following items contains four (4) alternatives, one of which provides the 

best answer. Circle the letter which precedes the best answer. 



1 . Who has the primary responsibility for determining what 
changes in job performance should occur? 

A. Trainer 

B. Line organization 

C. External training resource 

D. Trainees 

2. The best way for a trainer to deal with the identification of 
training needs is to 

A. periodically circulate check-off lists of courses 
available. 

B. wait for line managers to initiate training requests 
because these requests reflect true problem areas. 

C. choose educationally sound courses and locate problem 
areas in the organization they will help solve. 

D. systematically and continually collect data to help 
identify problem areas. 

3. Tailoring programs from external resources to specific local 
needs is easiest 

A. with prepackaged media presentations. 

B. for traditional courses in which objectives are not 
specifically defined. 

C. when training needs and course objectives are defined 
behaviorally. 

D. for computer-aided instruction. 

4. In deciding on class size for programs provided by external 
resources, 

A. it is often necessary to compromise between the small 
class size desired by line managers and the large classes 
desired by the external resource. 

B. facilities and equipment should not be allowed to limit 
class size. 

C. classes should be as large as physically possible so all 
trainees can be accommodated in the fewest sessions. 

D. it is often necessary to compromise between the large 
class size desired by line managers and the small classes 
desired by the external resource. 
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THE SELF-PACED LEARNING ENVIRONMENT 



INTRODUCTION 

A self-paced learning environment, or learning center, is more than a physical location; it 
is the embodiment of the fundamental principles of learning. In most cases, a learning center 
requires both a computer system and carefully designed instructional materials in various forms. 

This information sheet introduces the basic characteristics necessary to maintain the 
optimum learning environment for trainees. The learning objectives are: 

1 . LIST three distinguishing characteristics of individualized instruction, without the 
use of an aid and with no errors. 

2. LIST three characteristics of self-paced instruction, without the use of an aid and 
with no errors. 

INFORMATION 

The characteristics of individualized instruction form the basis of the self-paced instruction 
that occurs in a learning center. The three distinguishing characteristics of individualized 
instruction are: 

1 . It is designed to recognize each trainee as a unique individual. 

2. It facilitates trainee admission to a course or program at numerous points in time. 

3. It enables trainees to choose the instructional media which best suits them. 
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To individualize instruction fully, the presentation of instructional media must be tailored 
to the individual trainee. Some trainees learn best by listening, others prefer visual images. Some 
trainees absorb things well by reading. Notwithstanding all these learning styles, most 
trainees learn by doing, trying things out and actively getting hands-on practice. 

Ideally, individualized instruction offers instruction in a host of media. Print, videotape, 
motion pictures, audio tapes, slides, and other instructional media would all be available. The 
purpose of the variety is to let all trainees select the medium or media that they find most suitable 
for their personal learning style. Apart from being ideal, providing instruction in all possible 
media is unrealistic. Such an approach would require that each lesson topic be developed in every 
possible medium. It would require that every trainee receive a custom-tailored course. Few, if 
any, organizations can provide the unlimited resources that would be involved in such custom 
tailoring. 

As trainees go from assignment to assignment, they should have a choice of instructional 
media. The choice need not be unlimited, however. Instead, media options are designed into a 
course or program based on the needs of the content and projections about the nature of the 
trainees. Considerations of training needs like these allow course designers to develop optimum 
media mixes that give most trainees the media that will serve them best. Moreover, offering 
media choices used by only a small number of trainees is not cost effective. 

The three characteristics of self-paced instruction that incorporate the most positive 
features of individualized instruction are: 

1. Trainees can progress from assignment to assignment at their own rate of speed, 
within reason. 
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2. Trainees may challenge (take) a test at any time. 

3. Trainees may complete a course or program in different amounts of time, 
depending upon variables such as their skill and knowledge. 

Under the first characteristic, trainee rates of progress are kept “within reason” by using 
Predicted Completion Time (PCT). PCT is calculated before trainees begin a course of program, 
based on factors which influence their entry level. Instructors use comparisons of PCT to actual 
time spent in a course to monitor trainee progress. 

REVIEW QUESTIONS 

Directions : Study the information on this sheet. Then, without the use of an aid: 

1 . List three distinguishing characteristics of individualized instruction. 

1.1 

1.2 

1.3 

2. List three characteristics of self-paced instruction. 

2.1 



2.2 



2.3 
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INDIVIDUALIZATION OF INSTRUCTION 



Pace, Content, Sequence, and Style of Instruction 



Instructional systems combine organized collections of 
subject matter content with appropriate procedures for presenting 
the content to bring about learning outcomes. In individualized 
systems of instruction, the pace, content, sequence, and/or style 
of instruction are adjusted to the needs of individual learners. 
Pace 

Pace refers to the rate at which individuals progress 
through instructional content. It may be individualized in 
either or both of two ways. First, pace may be individualized by 
allowing individuals to proceed as rapidly as they can comprehend 
the material or as slowly as they wish through a set of 
instructional items. 

Second, different numbers of items may be presented to 
different individuals. In this situation, items may be randomly 
generated by computer or the instruction may involve simulation 
of a device or situation. It may also occur when items are 
repeated or sampled from a fixed quantity. When different 
numbers of items are presented, pace may be further 
individualized through the rate with which new instructional 
content or levels of difficulty are introduced. 
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Content 

Content is usually individualized through a diagnostic- 
prescriptive process. The individual's knowledge and skills are 
assessed prior to initiating work in any component of 
instruction. Components may range from the entire program of 
instruction, to a unit, or to a single item. Based on the 
results of the assessment, content is diagnostically adjusted to 
the individual's needs by prescribing that he/she skip the 
component entirely, study remedial material before working on it, 
or begin work in one or more "tracks" within the component 
itself . 

The assessment and instructional stages are not necessarily 
distinct. In some implementations, as the individual receives 
instruction, procedures built into the system adjust the content 
and its presentation in accordance with that individual's 
performance and estimated knowledge. 

Sequence 

Individualization of sequence can occur at two levels. At a 
macro level it concerns the order of topics addressed by the 
instruction. Implementations at this level may involve 
individual control, whereby the sequence of topics is determined 
by each individual learner. Alternatively, the instructional 
sequence may be determined in conference (s) between the 
individual and instructor or other system coordinator, or it may 
be determined solely by established procedure or algorithm. 
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At a micro level, individualization of sequence determines 
the order in which items are presented. The sequencing of items 
may be unique for each individual, but it is not tailored to 
personal needs or interests. Applications at this level may 
simply present items in an arbitrary or random order. 

Style 

The manner in which different individuals receive, process, 
store, and recall information may differ. These differences have 
been called cognitive styles. Instructional systems that try to 
adjust to these differences are said to provide individualization 
of style. In practice, individualization of style usually 
involves adjusting the method of delivery. For example, 
instructional content may be presented using printed materials 
alone, audiovisual media alone, or some combination of the two 
that is consistent with the cognitive style of the individual. 

Individualization of style is not as widely utilized as 
individualization of pace, content, and sequence. In addition, 
the literature suggests that it is less expensive and more 
efficient for individuals to adjust their learning style to 
instruction than for instruction to adjust its style to learners, 
but research has not yet settled the issue. Brent (1990) and 
Federico and Landis (1984) summarize much of the research on 
cognitive style and report data suggesting that individualization 
of style is beneficial to learners, but that the extent of these 
benefits depends on the learning objectives and instructional 
content . 
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Recapturing the Effectiveness of Tutoring 
Without the Cost 

The practice of grouping students together in a classroom 
took hold during the Industrial Revolution. While group 
instruction made mass education and training economically and 
administratively feasible, it was at the expense of the 
individualized tailoring of instruction provided by one 
instructor working with one student. Bloom (1984) characterized 
this issue as the "2-sigma problem." He found that when 
instructional time is held constant, students in a conventional 
classroom (with about 30 students and one teacher) score about 
two standard deviations lower than students given individual 
tutoring. 

Bloom defined the challenge for individualized instruction 
as one of recapturing the two standard deviations of achievement 
lost by group instruction. Accordingly, individualized systems 
of instruction can be viewed as attempts to recapture the 
effectiveness of individual tutoring without the expense. 
Technological Contributions 

Technology's first significant contribution to instruction 
occurred in the 1400s with Johannes Gutenberg's development of 
movable type and printed books. These developments made 
instructional content affordable to large numbers of people. 
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A second significant contribution of technology to 
individualized instruction occurred with the development of 
stored-program computers. Computers have made not only the 
content but also the interactions of effective instruction 
inexpensively available to large numbers of people. 

Computers implement the procedures for presenting 
instructional content that are key to individualization of 
instruction. However, Clark (1983) reminds us that "computer- 
based instruction" is too diffuse a term to describe any 
particular system or approach to instruction. 

The following discussion considers print-oriented and 
computer-oriented approaches to individualized instruction. 
Print-oriented approaches emphasize the use of paper (hard copy) 
materials, while computer-oriented systems capitalize on the 
promise of computer technology. Both approaches may use printed 
materials; the differences between the two primarily concern the 
amount of work performed by computers. 

Individualized Instruction: Print-Oriented Approaches 

Print-oriented approaches, such as modules, divide 
instructional content into units of instruction. Then, for each 
unit, they provide a pre-assessment (pre-test) , an individualized 
prescription and presentation of instructional content based on 
the pre-assessment, the instruction itself, and a post-assessment 
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(post-test) . Students must attain criterion levels of knowledge 
and/or performance on the post-assessment before progressing to 
the next unit of instruction. 

Programmed Instruction 

Development of programmed instruction was directly 
influenced by B.F. Skinner's paper, "The Science of Learning and 
the Art of Teaching" (1954) . His "extrinsic programming" breaks 
instructional material into a linear series of steps requiring 
learners to make active responses for which they receive 
immediate feedback. The steps are intended to be so small that 
the individuals' responses are almost always correct. Neither 
the content, sequence, nor style of presentation are adjusted for 
individual learners, but individuals determine the pace at which 
they complete the items and progress through the material. 

Despite Skinner's impact on programmed instruction, most 
applications of this approach are closer to the "intrinsic 
programming" described by Crowder (1962) and commonly used in 
programmed texts and tutorial computer-based instruction. 
Intrinsic programming is a pragmatic compromise between the 
difficulties and expense of devising Skinnerian extrinsically 
programmed materials and the inflexibilities of group 
instruction. 

Intrinsic programming permits larger instructional steps 
than extrinsic programming, but it still emphasizes active 
responding and immediate feedback to learners. All responses, 
correct or incorrect, can be examined and used to determine paths 
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for continuing instruction. Intrinsic programming requires 
instructional developers to anticipate the wrong responses likely 
to be made by individuals and the remedial material needed to 
correct the misconceptions and provide information they lack. In 
this way, intrinsic programming can support individualization of 
pace, content, sequence, and style. 

Hartley (1977), Kulik, Cohen, and Ebeling (1980), and Kulik, 
Schwalb, and Kulik (1982) have all reviewed research findings on 
the effectiveness of programmed instruction, although they did 
not distinguish between extrinsic and intrinsic programming. 
Hartley reviewed results from 89 studies of elementary and 
secondary school mathematics instruction and reported an average 
improvement of 0.11 standard deviations through the use of 
programmed instruction. 

In a review of 57 studies, Kulik et al. (1980) reported that 
programmed instruction used in higher education to present a 
variety of subjects improved performance by about 0.24 standard 
deviations over conventional instruction. In a review of 47 
comparisons, Kulik and others (1982) reported that programmed 
instruction used in secondary education to present a variety of 
subjects improved performance by about 0.08 standard deviations. 
Overall, these results suggest that the positive impact of 
programmed instruction is genuine, but limited. 

Programmed Texts and Teaching Machines 

After instructional content has been programmed, it must be 
presented to the student by means of some type of "holder," such 
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as a programmed text or teaching machine. Programmed texts and 
teaching machines are, however, limited primarily to teaching 
knowledge-level material. Teaching manipulative skills requires 
the use of actual job equipment or a simulator. It is also 
important that skill training take place in an environment which 
duplicates the operational environment as closely as possible. 

Many authorities are convinced that a programmed text is 
sufficient for most programmed instruction. Such texts have been 
developed for a wide variety of subjects, including mathematics, 
electricity-electronics, foreign languages, accounting, 
management, computers and data processing, and medicine. 

A teaching machine is a mechanical or electronic device that 
presents instructional content to students on an individual 
basis. Students have their own machines and receive instruction 
as fast as they can absorb it. Assuming, of course, that the 
student needs the instruction to begin with, machines provide 
instruction which is both individualized and self-paced. 

Teaching machines are produced with varying degrees of 
complexity and special features; however, they all represent some 
form of tutorial teaching. They present each student with 
problems, exercises, questions, and answers. 

The difference between teaching machines and most other 
individual instructional methods is that teaching machines 
provide immediate feedback to students. Errors can be corrected 
immediately because students are made aware of their progress 
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during each phase of instruction. Thus, teaching machines differ 
from other media in three ways: 

1. Students must respond continuously and actively, 
providing explicit practice and testing of each step of what is 
to be learned. 

2. Students receive immediate feedback on their responses, 
so that errors can be corrected. 

3 . Students proceed on an individual basis at their own 
pace — faster students advancing rapidly through an 
instructional sequence, slower students being tutored as slowly 
as necessary, with infinite patience to meet their special needs. 

The nature of teaching machines and programmed instruction 
provides a channeling of student activity into a relatively 
narrow chain of experiences. Only a small portion of the lesson 
is available at any one time, so students cannot advance or 
regress unless that action is programmed into the lesson or 
machine. 

In presenting a program, a teaching machine provides 
immediate feedback on all student responses. Unlike a programmed 
text, students cannot see the feedback before responding. 

Students are therefore obliged to subject themselves to all the 
conditioning designed into the programmed lesson. 

Teaching machines usually maintain a record of student 
performance. This record can be used to analyze progress and 
serve as a basis for determining future studies. Collectively, a 
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group of student records may be used to determine what curriculum 
changes are required. 

Some feel that the teaching machine provides a constant 
motivational factor to the student. The student is challenged to 
"beat the machine." Others claim that this fascination wears off 
after a time, and once it does, the machine is no more effective 
than a programmed text (AFM 50-62, 1984, p. 19-5). 

Personalized System of Instruction 

Keller's Personalized System of Instruction (PSI) was 
disseminated by his 1968 article, "Goodbye, teacher . . . . " PSI 
has been used primarily to replace lecture-based, classroom 
teaching in higher education. Keller listed five features that 
distinguish PSI from other instructional systems: (a) the unit 

mastery requirement, (b) student self-pacing, (c) student 
proctors, (d) reliance on written instruction, and (e) de- 
emphasis on lectures. 

Like Crowder's intrinsic programming, Keller's PSI uses 
larger steps than those recommended for Skinner's extrinsic 
programming. It separates instructional content into content 
units that are presented in a linear sequence, and it requires 
students to demonstrate mastery of each unit before proceeding to 
the next. 

Unlike intrinsic programming, PSI leaves most of the 
individualization up to the individuals. In place of the within- 
unit instructional items of intrinsic programming, learners 
receive study guides. Each study guide introduces its unit, 







200 



220 



11 

lists learning objectives, suggests instructional resources for 
attaining the objectives, and recommends study questions to help 
students prepare for the mastery examinations. 

PSI takes a hint from peer tutoring in its use of proctors 
to guide students. The proctors are generally recent graduates 
of the course and are chosen for their "mastery of the course 
content and orientation . . . maturity of judgement . . . 
understanding of the special problems that confront [beginners] 
and ... willingness to assist" (Keller, 1968, p. 81). Keller 
suggests that the use of proctors is the major innovative 
component of his approach. 

Perhaps the most comprehensive review of PSI effectiveness 
is a meta-analysis documented by Kulik, Kulik, and Cohen (1979a) . 
These researchers reported that the PSI programs studied raised 
final examination scores by about 0.50 standard deviations over 
programs using conventional (non-PSI) instruction. They also 
found that PSI produced less variation in achievement, higher 
student ratings, and fewer course withdrawals, and that these 
favorable results occurred across a variety of subject matters 
and course settings. 

Nevertheless, Keller, writing in 1985, was pessimistic about 
the future of PSI. Problems he cited as most prevalent were the 
investment of instructor time needed to set up PSI courses and 
the general lack of support from university administrators. 

Lloyd and Lloyd (1986) corroborated his concerns. They reported 
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that progressively fewer PSI courses are being taught and many of 
those depart substantially from the recommended PSI format. 
Audio-Tutorial Approach 

The audio-tutorial (A-T) approach resembles PSI in that it 
also modularizes instructional content into units, is mostly 
applied in higher education, and leaves much of the 
individualization up to the students. Its basic form was 
developed in the early 1960s by Samuel Postlethwait and later 
described by Postlethwait, Novak, and Murray (1972). It consists 
of (a) individual study sessions using audio tapes and/or other 
self-study media; (b) weekly group assembly sessions for 
lectures, films, and major examinations; and (c) small group 
(6-10 students) quiz sessions. In its early form, the sequence 
of content units was linear and each unit was intended to 
encompass about a week's work, limiting individualization of both 
pace and sequence. 

In later forms, A-T evolved into the concept of mini- 
courses, self-contained modules of varying length that can be 
presented in a variety of sequences. The mini-courses resemble 
A-T content units in that students begin with a study guide and a 
list of learning objectives and then proceed to the usual A-T 
individual study sessions, group sessions, and small group work. 
After an established period of time, their knowledge of the unit 
is assessed by an examination given in a group session. 

As with PSI, the most thorough assessment of A-T was 
performed by Kulik and his associates (Kulik, Kulik, & Cohen, 



ERIC 



090 

^ M 



202 



13 



1979b). In summarizing the results of 42 studies, Kulik et al. 
reported that A-T increased overall student achievement by about 
0.20 standard deviations over conventional means of instruction. 
This finding held up over a variety of subject matters and higher 
education settings. Thus, the overall positive impact of A-T, 
like programmed instruction, appears to be genuine, but small. 

Individualized instruction: Computer-Oriented Approaches 

Computer-oriented approaches can provide individualization 
of pace, content, sequence, and style of instruction. These 
approaches have been reviewed and were found to be effective 
(Niemiec & Walberg, 1987). Fletcher (1991), however, has 
suggested that their effectiveness may be significantly increased 
by including multimedia capabilities. 

Computers can be used directly to teach, as in computer- 
assisted instruction (CAI) , and to manage instructional 
processes, as in computer-managed instruction (CMI) . CAI and CMI 
are often presented as contrasting approaches. However, CMI may 
include CAI among its prescriptions, to manage student progress; 
and both may be used to support individualized systems of 
instruction. 

Of the computer-oriented systems discussed here, 

Individually Prescribed Instruction (IPI) , the Adaptive Learning 
Environments Model (ALEM) , and the program for learning in 
accordance with needs (Project PLAN) use computer resources to 
help manage instruction. The strands approach, optimized 



O 

ERJC 



203 



223 



14 

instruction, and intelligent computer-assisted instruction use 
computers directly to teach. 

Individually Prescribed Instruction 

Individually prescribed instruction (IPI) was developed in 
the mid-1960s at the University of Pittsburgh's Learning Research 
and Development Center as an individualized system of 
instruction. As with the PSI and A-T approaches, development and 
use of learning objectives are emphasized and content is 
modularized into units. Each content unit is preceded by a pre- 
test, which determines if the unit objectives have already been 
attained by individual students. Results of the pre-tests are 
then used to prescribe the instructional content that individuals 
are to study. A post-test is administered at the end of each 
content unit. As in PSI, individuals must demonstrate mastery of 
each unit's learning objectives on the post-test before 
proceeding. 

Adaptive Learning Environments Model 

IPI evolved into the adaptive learning environments model 
(ALEM) , which was described by Wang and Walberg (1983). ALEM 
combines aspects of prescriptive instruction from IPI with 
independent inquiry and social cooperation from open education. 

It allows greater adjustment of content than does IPI for 
individuals with remedial needs by providing tailored learning 
activities for them. It allows for individualization of pace, 
content, sequence, and style of instruction. 



ERJC 



n r> 



4 



204 



15 

ALEM includes an organizational structure to help schools 
implement its essential components. This structure consists of 
(a) a basic skills curriculum of both highly structured and open- 
ended activities, (b) an instructional management system that 
guides use of instructional time and material resources, (c) a 
family involvement component to integrate school and home 
learning, (d) a system to devise flexible grouping of students 
and team teaching, and (e) a staff development component to aid 
monitoring and implementation of the program. Its central 
objectives are time related. ALEM is intended to reduce learning 
time, while increasing the time actually spent on learning and 
instruction. 

Program for Learning in Accordance with Needs 

Like IPI and ALEM, the program for learning in accordance 
with needs (PLAN) used computer resources to manage instruction. 
It also emphasized the development of learning objectives. It 
was developed in the late 1960s by the American Institutes for 
Research, Westinghouse Learning Corporation, and 14 school 
districts (Flanagan, Shanner, Brudner, & Marker, 1975) , but fell 
into disuse, for a variety of reasons, in the late 1970s. 

PLAN included about 6000 learning objectives for language 
arts, mathematics, and science in Grades 1-12. The instruction 
was divided into teaching-learning units (TLUs) that were 
developed for each learning objective. A computer facility was 
used to collect, store, and process information on student 
performance and progress. 



ERIC 




205 



16 



Like most classroom systems for individualizing instruction, 
PLAN used diagnostic-prescriptive procedures to determine what 
instructional content to present. Periodically, a teacher and an 
individual student would decide on a subset of objectives that 
the student was to attain next, based on the student's school 
history and placement test results, which were stored in the 
computer system. The student was then assigned objectives and 
given a TLU study guide for each. Each TLU study guide 
identified instructional resources and activities to be used in 
attaining its objective. Unlike IPI, these resources were almost 
always commercial materials that were not specifically developed 
for this system. PLAN included tests to help teachers assess 
student progress and a program to train teachers and 
administrators in its proper use. 

Strands Approach 

Computer-assisted instruction (CAI) may be divided into four 
categories: 

1. Drill and practice — primarily supported by the strands 
approach; involves the presentation of relatively discrete items 
to students for the purpose of practice 

2. Tutorial CAI — generally based on and differs little 
from Crowder ian intrinsic programming (described under the 
heading Programmed Instruction) ; resembles programmed instruction 

3 . Tutorial simulation — a computer is used to emulate a 
device or situation; typically found in training applications of 
CAI, but increasingly common in educational applications 
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4 . Tutorial dialogue — an attempt to directly incorporate 
the features and benefits of one-on-one, teacher-student 
interaction in a computer program 

Tutorial dialogues, frequently in combination with tutorial 
simulation, are implemented using intelligent computer-assisted 
instruction. 

The strands approach has been widely used in computer 
curriculum development since it was conceived at Stanford 
University in the 1960s. 

The term "strand" identifies a basic component skill of the 
instructional content. For instance, letter identification is a 
strand in beginning reading instruction and single column 
addition is a strand in beginning arithmetic instruction. 

Progress within strands is criterion dependent. Students proceed 
to a new exercise or new instructional content within a strand 
only after they attain some performance criterion in the current 
exercise, thereby providing a running assessment of their 
progress that can be examined by teachers and system 
administrators at any time. Students usually work in several 
strands during a single session, and branching between these 
strands is time dependent. Students move from one strand to take 
up where they left off in another strand after an individually 
specified amount of time. Initial entry into a strand depends on 
progress in other strands. The strands approach allows for 
individualization of pace, content, sequence, and style of 
instruction. 
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No overall review of the effectiveness of the strands 
approach was found in the literature, but assessments of 
individual CBI programs using it have reported favorable results. 
Suppes, Fletcher, and Zanotti (1976) showed that time spent in 
strands mathematics could predict instructional progress to the 
nearest tenth of a grade placement in standardized tests. 

Ragosta et al. (1982) reported an average improvement of 0.26 
standard deviations in achievement for strands applications in 
mathematics over less-individualized instruction approaches. 
Intelligent Computer-Assisted Instruction 

The development of individualized programs of instruction 
that apply computer-based representations of knowledge to the 
problems of instruction has been directly supported by the 
emergence of cognitive science. These approaches provide 
powerful means for individualizing pace, content, sequence, and 
style of instruction. 

Intelligent computer-assisted instruction (ICAI) may be as 
unintelligently designed as any other approach to instruction. 
"Intelligence", in this case, suggests an attempt to apply 
knowledge-based or information-structure-oriented (ISO) 
approaches to the processes of instruction. Carbonell (1970) 
contrasted these ISO approaches with ad hoc frame-oriented (AFO) 
approaches based on programmed instruction techniques such as 
intrinsic programming. 

AFO approaches depend on preprogrammed blocks of static 
material usually called frames. These frames consist of 
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instructional material followed by questions. Associated with 
the questions are prespecified correct and incorrect answers that 
are matched with students' responses. Students have no 
opportunities to ask questions unless they match the prespecified 
query options provided by the system developers. 

The ISO approach is based on a knowledge representation of 
the subject matter. A separate tutorial program operates on the 
knowledge representation, in order to generate instructional 
material, including questions and the answers to them, and 
responds to inquiries initiated by students. Discussions 
concerning the subject matter can be initiated either by the 
computer program or by the student. This capability for mixed 
initiative is characteristic of all intelligent computer-assisted 
instruction. This approach depends on the generation of material 
from the knowledge base rather than on preprogrammed material and 
pre-stored, anticipated student responses. To accomplish its 
objectives, an ICAI program must include three capabilities. It 
must represent 

1. The relevant knowledge domain, including both a subject 
matter expert's understanding and that expert's ability to 
explain it to learners (Brown, Burton, & deKleer, 1982) . 

2. The student's state of knowledge, or model, of the 
subject matter, providing both diagnostic support for instruction 
and a representation of the student's misconceptions (Clancey, 
1986) . 
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3. An expert tutor, providing the means to sustain the 
student from one state of knowledge to another (Halff , 1988) . 

Wenger (1987) provided a comprehensive overview of ICAI 
systems and Psotka, Massey, and Mutter (1988) described many 
specific implementations of ICAI. Despite the number of years 
ICAI systems have been in development, information on their use 
in instructional settings is only beginning to emerge, and few 
conclusions can currently be drawn about their effectiveness. 
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PROGRAMMED INSTRUCTION 



The three methods of programmed instruction in common use are (a) linear, (b) adjunct, and 
(c) branching. This handout was prepared to familiarize you with the major characteristics of 
each type. 



Linear Programming 

To use this linear program, first cover the right-hand column with a piece of paper. Read 
step 1 and write your answer on the blank line(s) provided. Then slide the cover-paper down the 
right-hand column just far enough to read the correct answer for step 1 . If your answer is correct, 
go to the next step and follow the same procedure. 



If your answer to any step is different from the one in the right-hand column, reread the step 
and correct your answer. Then go to the next step. 



1 . The subject matter in a programmed text using the 
linear technique is presented in small steps. 

Learning is easier when you study new material in 

. small steps 

2. Each small step is designed to help you make the 
correct response. Blank spaces are provided for 

your . responses or answers 

3. When learning a new subject, trainees often make 
many errors. If a linear program is carefully 
prepared, you should make (many/few) errors. 

few 



4. Your answers are called "constructed responses" 
because you write them out. Your response to each 

in a linear program 

is called a . 

5. You only respond to material that is to be learned. 
When you write out a response, it is called a 



6. So far you have learned two characteristics of a 
linear program. A linear program is written in 

and the method of 

responding is called 



small step 

constructed response 



constructed response 



small steps 
constructed response 
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7. Coaching techniques, called prompts or cues, are 
sometimes used to increase the probability of a 
correct response. . 

A word that is underlined is a prompt or cue. 
Underlining of the two words in the statement 
above gives assistance, so you can respond to the 
words p or c . 

8. Gradually the prompts and cues are withdrawn, in 

successive steps, as you become more self-reliant 
and master the subject matter. After a response has 
been learned, the prompts or cues are . 

9. A third characteristic of a linear program is 

"knowledge of results." In a linear program, 
acceptable answers are written in the right-hand 
column of the text. Each time you make a response 
you are given immediate 



1 0. An answer remains concealed from view until a 

response is made. Then the answer is revealed and 
your response is reinforced by giving you 
immediate . 

1 1 . When difficult subject matter is broken down into 
small parts so that you can easily go from one step 
to the next, you are using the characteristic of 



12. When the program from which you are learning 

demands an answer to be written out, you are using 
the characteristic of . 

1 3. Immediate confirmation of a right or wrong answer 
describes the characteristic called 



14. Remember the three characteristics of a linear 

program. It is written in , 

the method of responding is called 

, and being immediately informed of a 

correct or incorrect answer is called 
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prompts cues 



withdrawn 



knowledge of 
results 



knowledge of results 



small steps 



constructed response 

knowledge 
of results 



small steps 
constructed 
response 
knowledge 
of results 
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You have just completed a linear program which familiarized you with three characteristics of 
linear programming. 

Other methods of programming use the same characteristics. However, their style of 
presentation and format are quite different. 

The next method of programming to be covered is "Adjunct." You will easily see how the 
format of an adjunct program differs from that of a linear one. 

We will use the adjunct method to familiarize you with the characteristics of an adjunct 
program. 
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Adjunct Programming 

Read the following excerpt from a training manual pertaining to Adjunct Programming. 

After familiarizing yourself with the information, answer the questions on this page. (The 
correct answers appear at the top of the next page.) 

Adjunct programming, in its simplest form, consists of a series of 
questions prepared as an adjunct to existing print-based (hard copy) 
materials, such as textbooks or training manuals. The questions are 
designed to elicit the type of responses called for in the learning 
objectives. For example, if the objectives call for trainees to discriminate 
between electronic components, questions may require them to select 
correct components from many alternatives. If the learning objectives 
state that trainees must be able to perform a task, questions may ask them 
to show how they would use the objects or processes involved. However, 
ability is measured by performance and, when possible, the questions 
require answers that actually demonstrate the attainment of learning 

objectives. 

1 . True or False: Adjunct programming uses existing hard copy materials. 

2. When you are directed to read a portion of a training manual and answer questions, 

this is a characteristic of what programming method? (Linear or Adjunct) 

3. Write the letter (A-D) of the phrase which best completes the following statement: An 

adjunct program 

A. consists of a series of small steps to elicit the type response called for in a learning 
objective. 

B. uses existing hard copy materials and questions to elicit the type response called 
for in a learning objective. 

C. is bulky and therefore not very practical for trainee use. 

D. requires trainees to respond frequently as they read the material. 

4. Write, in your own words, the characteristics of an adjunct program. 
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ANSWERS 

1 . True 

2. Adjunct 

3. B 

4. Uses existing hard copy materials and questions to elicit the type response called for in a 
learning objective. 

Branching Programming 

Another method of programming is called "Branching." You will be introduced to its style 
of presentation and format by using a short example of a branching program. 

Each of the following steps represents a page from a branching program. 

Begin here with Page 1. (Follow instructions carefully.) 



Page 1 

A branching program differs from a linear or adjunct program in several ways. Nevertheless, 
the trainee is still required to respond to new material. The major difference in a branching 
program is that a response to a question directs the trainee to alternate sequences or branches 
of material. This allows the trainee to make mistakes, and gives remedial information when a 
wrong answer is chosen. 

Choose the correct statement from below and turn to the page indicated. 

A. A correct response would direct a trainee to the next logical item of 
information, thereby omitting remedial information. 

TURN TO PAGE 3 

B. A branching program forces the trainee to make mistakes. 

TURN TO PAGE 2 
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(From Page 1) 



Page 2 



Your answer: B. A branching program forces the trainee to make mistakes. 

This is not entirely true. A branching program makes allowances for 
individual differences in experience and learning, and provides necessary 
remedial information, but it generally does not force a trainee to make 
mistakes. 



Your answer: A. A correct response would direct a trainee to the next logical item of 
information, thereby omitting remedial information. 

You are correct. Trainees with a good background of the information being 
presented would not require any additional instruction. They continue through 
the program in a straight path, coming in contact only with "right-answer 
pages." Here the answer is confirmed and new material is presented. When 
trainees select a wrong answer, they are directed to a wrong-answer page. 



Which of the following statements is correct? 

A. A wrong-answer page gives remedial instruction. 

TURN TO PAGE 5 

B. A branching program requires all trainees to follow one path. 



Return to Page 1 ; re-read the information and 
select the correct statement. 



(From Page 1) 



Page 3 



TURN TO PAGE 4 
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(From Page 3) Page 4 

Your answer: B. A branching program requires all trainees to follow one path. 

You have chosen a wrong answer. It is possible for some trainees, but not all 
of them, to proceed through a branching program in a single path. Those that 
choose wrong answers are given additional instruction on wrong-answer pages. 

A wrong-answer page gives remedial instruction. 

TURN TO PAGE 5 and read the correct answer. 



(From Page 3 or 4) Page 5 

You have arrived at this page for one or two reasons. You came here either straight from Page 
3 or after proceeding through the remedial instruction on Page 4. 

Correct answer: A. A wrong-answer page gives remedial instruction. 



A wrong-answer page supplies a trainee with more background knowledge on a subject, or 
explains the original item in a different way. The wrong-answer page may also clarify some 
misconceptions that the trainee might be having. When a mistake has been clarified, the 
trainee is sent back to the original question and asked to make another selection or sent on to a 
new page of information. 



This concludes the sequence on the characteristics of a branching program. 

(Turn to the next page.) 
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If you went through the sequence of steps without reading the wrong-answer pages, 2 and 4, 
you got through without making a mistake. If you selected an incorrect response, you were 
branched to a wrong-answer page and given additional instruction. This illustrates how a 
branching program is designed. 

Answer the following questions. (The correct answers appear at the top of the next page.) 

1 . The characteristics of a branching program that distinguish it from a linear or adjunct 
program are: (circle the letter which precedes the best answer.) 

A. A branching program is not adaptable to trainees with individual differences. 

B. The linear and adjunct programming methods are similar to that of branching. 

C. The wrong-answer pages give remedial information. 

D. Pages in a branching program are usually written in textbook style. 

2. List the major characteristics of a branching program. 
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ANSWERS 

1. C 

2. • The response to a question directs you to a right-answer or wrong-answer page. 

• Wrong answer pages give remedial information. 

• Right-answer pages confirm the correct answer and present new material. 



The end. 
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RESEARCH ON PREPARING SELF-INSTRUCTIONAL MATERIALS 



Charles W. Howard examined (a) the effects of five strategies for preparing self- 
instructional materials, and (b) whether existing instructional materials could be adapted into 
self-instructional materials which promote learning. The primary purpose of Howard’s study 
was to determine the efficiency of five different strategies for preparing self-instructional 
materials. Efficiency was investigated in terms of achievement and time. 

The five strategies for preparing self- instructional materials were: 

1 . Instructional text 

2. Instructional text supplemented by post-text questions 

3. Instructional text supplemented by post-text questions, along with knowledge 
of results of the post-text questions 

4. Instructional text supplemented by post-text questions, along with knowledge 
of results of the post-text questions, and directions for remedial instruction 

5. Programmed instruction, developed using content identical to that of the 
instructional text, and containing linear and branching programming 

Methodology 

The population from which the research sample was drawn consisted of U.S. Army 
enlisted and commissioned engineering personnel. The sample consisted of 720 subjects who 
were assigned to various technical training programs. Subjects were randomly assigned to 
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one of six groups, i.e., five treatment groups and one control group. This random 
assignment yielded 120 subjects per group. 

Pre-test scores were used to determine each subject’s level of mastery. Each was 
classified as a non-master and therefore admissible as a subject for this study. 

The primary component was the instructional text content (item 1 on the previous 
list), which identified and described the functions of parts for a low-voltage circuit tester. 

The second component consisted of post-text questions derived from the content. The 
questions were designed to facilitate learning for knowledge and comprehension of the 
content. Knowledge of results (KR), the third component, consisted of subject matter 
experts’ responses to the post-text questions. For purposes of the study, distinctions between 
the strategies were made by providing KR to the learners in three of the five strategies. The 
fourth and last component, remediation, consisted of a written statement directing learners to 
appropriate portions of the content when their actual performance was not the same as the 
model performance. 

The administration of each of the five strategies followed a common procedure. 

1. Instructional Text: Subject was given a manual that contained the content, 
with no programming. 

2. Text/Questions: Subject was given a manual that contained the content 
material plus a set of questions which were at the end of the material. The directions 
instructed the learner to overtly respond to the questions. 
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3. Text/ Questions/Knowledge of Results: Subject was given a manual and a set 
of questions as in item two, except correct responses for the questions were included, in 
addition to the text/questions. 

4. Text/Questions/Knowledge of Results/Remediation: Subject was given a 

manual containing the content, questions, knowledge of results, and directions to portions of 

* 

the text to review non-mastered content. 

5. Programmed Instruction: Subject was given programmed (self-instructional) 
materials. 

6. Control: Subject was pre-tested and post-tested with no instructional 
treatment. 

The pre-test and post-test consisted of 27 matching items. The format was to provide 
each subject with a list of statements and a graphic representation of the low-voltage circuit 
tester. The graphic included the parts of the circuit tester pictured with numerics. The 
objective was to match the statement with the numeric in the graphic. 

The transfer test, consisting of 21 completion items, was designed to measure the 
transfer of knowledge and skills presented in the materials to their application. A graphic 
workbook was substituted for the actual equipment. The format was to provide each subject 
with a written statement of the conditions and standards and a graphic representation of the 
equipment. The subject was then required to indicate the (a) appropriate hook-up, (b) range 
of readings, and (c) trouble-shooting procedure. 
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Results 

The dependent variables post- test time, total package time, and total teaching time 
were analyzed by an analysis of variance model (ANOVA). The results indicated that the 
instructional text plus question group was not significantly higher than the programmed 
instruction group in post-test time, total package time, or total teaching time. 

Therefore, a strategy was identified for preparing self-instructional materials that is as 
efficient as programmed instruction. 

Evaluation of the effectiveness of the strategies was measured in terms of 
performance. The dependent variables pre-test and post-test performance scores were 
analyzed. The results indicated that the instructional text plus question group was not 
significantly different than the programmed instruction group. Thus, a strategy was 
identified for preparing self-instructional materials that is as effective as programmed 
instruction. 

The dependent variable transfer test score was analyzed. The results indicated that 

transfer of skills and knowledge occurred as well as in programmed instruction with 
three other strategies: (a) text plus questions; (b) text, questions, plus feedback; and 
(c) text only. 

Findings suggest that there is a strategy for preparing self-instructional materials that 
is as effective and efficient as programmed instruction. Programmed instruction, which 
results from a rigorous application of instructional systems development procedures, is 
extremely time-consuming and therefore expensive. 
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Conclusions 

This research examined (a) the transfer of cognitive skills; (b) the effects of KR, 
remediation, and post-text questions; and (c) informational feedback. The investigation 
focused on the effectiveness of five strategies for preparing self-instructional materials and 
the utility of informational feedback. 

The results support post-text questioning. Instructional text or written material 
followed by post-text questioning resulted in performance as effective as programmed 
instruction. Post-text questions are therefore regarded as aids which promote learning. 

Although it seems unusual, the research found that two treatments — text plus 
questions plus feedback, and text plus questions plus feedback plus remediation — when 
uncontrolled, were less effective than programmed instruction or the use of text plus post-text 
questions (with no feedback or remediation). 

Informational feedback was provided to 50% of the subjects in each of the six groups. 
Informational feedback in the form of KR was not found to significantly affect the 
performance of subjects on the transfer test. 

In summary, the results support the following conclusions: 

1 . The use of post-text questions in the form of instructional text plus post- 
text questions was demonstrated to be the most effective alternative strategy. 

2. Informational feedback in the form of post-text KR was not found to 
significantly affect the performance of subjects on the transfer test. 

3. Knowledge of results plus remediation, when uncontrolled, was less effective 
than the use of post- text questions only. 
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Implications and Recommendations 

An important implication of this study is that post-text questions create a learning 
situation that is both meaningful to the learner and desirable to the instructor and the 
institution. Use of post-text questions requiring overt response without any form of 
feedback produced the most desirable results in terms of both time and performance for 
the alternative strategies. 

This implication has major impact for instructional materials developers. Arguments 
against it would generally indicate that treatment groups receiving instructional text plus 
questions plus feedback would presumably obtain higher scores. However, as demonstrated 
within this study, this was not the case. Therefore, instructional materials for which 
treatment groups do not have access to the correct answers, but are permitted to review 
the instructional materials, are both effective and efficient. Additionally, existing 
instructional materials can be adapted into self-instructional materials which promote learning 
by adding post-text questions. 
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TRAINING COSTS & BENEFITS 



Directions: Each of the following items contains four (4) alternatives, one of which provides the 

best answer. Circle the letter which precedes the best answer. 



1 . Financial benefits are typically easiest to measure in 
programs to train 

A. first-line supervisors. 

B. production skills. 

C. instructors. 

D. management practices. 

2. A recommended procedure for achieving cost-effective 
training is to 

A. determine the absolute minimum training needs and 
deliver low-cost programs to meet these. 

B. always use the lowest-cost training method available. 

C. design and estimate costs for an optimum program 
and compare alternatives with this. 

D. concentrate on training costs since benefits cannot be 
measured in financial terms. 

3. The package presented by the training department to 
support its operating and capital budgets usually should 
include 

A. training needs, programs, costs, and benefits. 

B. information on scheduling and participant costs. 

C. no mention of objectives other than financial ones. 

D. substantial excess costs to allow for cutting. 

4. What is the advantage of charging the costs of a training 
program back to the using department or organization? 

A. The training department gains control of some of the 
operating department's funds. 

B. It automatically establishes the financial benefits of 
the training program. 

C. It substantially reduces the total cost of training to the 
overall organization. 

D. It reduces the training department budget by 
transferring some costs to the users. 

251 



NOTES 



231 



Human Resource Development Department 



The University of Tennessee 



DETERMINING THE COST EFFECTIVENESS OF TRAINING 



Learning Objectives 

Upon successful completion of this module, you will be able to: 

1. Determine when to measure and report the cost effectiveness 
of training 

2. Calculate the costs of training 

3. Describe four levels of measuring training effectiveness 

4. Select the outcomes (benefits) to be measured and link 
training to those outcomes 

5. Compute how much training returns relative to its cost 
(return on investment) 

6. Justify training investments based on the cost-benefit ratio 

'7. Compute the value added by training, using the bottom-line 

evaluation method 

8. Apply the payback period method to determine how long it 
will take for training to pay for itself 

9. Describe the advantages and disadvantages of four different 
methods of determining cost effectiveness 

10. Apply four methods to determine if a training program is 
cost effective 

Rationale (purposes) 

1. To discuss the need for justifying training expenditures 
with documented benefits 

2. To provide details on (a) calculating training costs, (b) 
measuring the effectiveness of training, and (c) methods used in 
determining the cost effectiveness of training. These methods provide 
quantitative evidence that effective training is a worthwhile 
investment rather than a cost. 

While this module focuses on the cost effectiveness of training 
by private sector organizations, the content is also applicable to 
training provided by public (government) employers. 
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Introduction 

Due to intense competition in an era of rapid economic and 
technological change, employers are closely scrutinizing their 
spending on workforce training. Increasingly, training departments 
and human resource professionals are being asked to justify whether 
training is a worthwhile investment. Training managers would prefer 
to measure the effectiveness of their courses and programs with data 
from criterion-referenced tests and feedback questionnaires. However, 
higher management requires that training be further justified in terms 
that are important to them (financial) . 

Organization of the module. This module is organized in three 
parts. The first part points out why some training managers are 
reluctant to determine the cost effectiveness of training. It 
discusses the need to justify training expenditures with documented 
benefits. Part 1 also provides practical details and examples of how 
to calculate the direct, indirect, and full costs of training. A 
progress check is provided to facilitate your understanding of the 
content covered and to promote learning. 

Part 2 describes four levels of measurement used to determine the 
effectiveness of training. In addition, the need for a cause-and- 
effect link between a specific organizational problem and a 
performance deficiency is addressed. This second part of the module 
also discusses the selection of outcomes (benefits) to be baselined 
before training and tracked after training in order to determine the 
payback. Like Part 1, there is a progress check to facilitate review 
of the content covered in order to promote learning. 

Part 3 of the module presents four methods for determining the 
cost effectiveness of training courses and programs. The methods 
described are: 

1. Return on investment (ROI) 

2. Cost-benefit ratio 

3. Bottom-line evaluation 

4 . Payback period 

In addition to providing details and examples for each of these 
methods, their advantages and disadvantages are identified and listed 
in a table. Furthermore, a progress check is provided for each 
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method. These progress checks will help you determine how well you 
understand each method. 

At the end of the module is a practical example to facilitate a 
skill check on your understanding and attainment of the learning 
objectives listed on page 1. 

A note on terminology. In order to enhance understanding, some 
definitions of the terminology used in this module require 
clarification. 

• Bottom-line evaluation. This method shows the value added 
to each trainee's job performance and the total value added to an 
organization from training. 

• Cost-benefit ratio. The ratio of projected costs of 
training to its estimated benefits. It is a useful method for 
justifying training investments when the benefits attributable to 
training are difficult to quantify in monetary units. 

• Cost effectiveness. The results attained against the costs 
of time, effort, money, and inconvenience. Cost effectiveness also 
suggests assigning quantitative values to performance improvements. 
The criteria for effectiveness are value, worth, and merit. 

• Payback period. A method for initial consideration of a 
questionable training investment. It answers the question, "How long 
will it take the training to pay for itself?". 

• Return on investment (ROI) . The rate of what something 
returns relative to its cost. It is a calculative approach to 
evaluating a result against the amount of resources invested. 
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Part 1 — Training Costs 

When to Measure Cost Effectiveness 

Employers have traditionally supported training because (a) it 
shows the organization's concern for its employees, and (b) higher 
management "assumes" that the benefits exceed the costs. In any 
event, when higher management "believes" that training is 
operationally critical to the organization's competitive position, 
there may be little or no demand for cost-effectiveness information. 

Therefore, despite the increasing admonishment found in training 
literature about the importance of demonstrating that training is a 
worthwhile investment, many training managers routinely avoid the use 
of economic justification. However, when a belief in the value, 
worth, and merit of training is not part of an underlying business 
philosophy, reporting the cost effectiveness of training can help the 
training manager establish credibility and may enhance the 
organization's willingness to invest additional resources. 

The single greatest incentive for indicating the cost 
effectiveness of training is its use in justifying training 
expenditures with documented benefits. Uncertainty about continued 
support for the training function leads to the consideration of cost- 
effectiveness information as a defensive measure — a way of showing 
higher management that training is not a luxury. Consequently, a 
growing number of training managers have a desire to show a return on 
training investments similar to that on other business investments. 
Nevertheless, many lack the knowledge necessary to determine the costs 
and benefits of training courses and programs (Lombardo, 1989) . 

When asked to report on a training investment, training managers 
often hesitate to calculate the costs necessary to develop, deliver, 
and evaluate training, and are reluctant to document and report the 
benefits such as increased quality, productivity, sales, and so forth. 
Among the reasons cited for this reluctance are the 

1. Lack of reliable cost figures 

2. Difficulty in identifying, monitoring, and quantifying 
training benefits 

3. Subjective nature of the assumptions to be made 
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4. Inability to isolate training's influence on performance 
improvements from other factors 

5. Time and effort involved in calculating the costs and 
documenting the benefits of training 

6. Potential for unfavorable returns on the investment 
Notwithstanding these and other reasons, training managers who 

feel the need to justify training with evidence that the benefits 
exceed costs will welcome the opportunity to determine the cost 
effectiveness of training using the methods presented in this module. 
These methods can provide quantitative evidence that effective 
training is a worthwhile investment rather than a cost. Once in 
place, the methods become easier to use each time they are applied. 



Calculating the Costs of Training 

Costs are incurred in developing, conducting (delivering) , and 
evaluating training. These costs are categorized as direct and 
indirect. The full cost of training is the sum of all direct and 
indirect costs. 

Often, the training manager will not have access to many of the 
direct and indirect costs of training and must obtain figures or 
reliable estimates from the organization's payroll, budget, 
accounting, or comptroller's office. Usually, however, all costs can 
be accounted for. 

Direct costs. Direct costs are expenses tied specifically to a 
product (training course or program) (Usry, Hammer, & Matz, 1988, 
p. 26) . Direct personnel costs include the wages and benefits paid to 
or on behalf of employees involved in training (e.g., trainees and 
instructors) as well as fees paid for professional services purchased 
from external providers (contractors, consultants, etc.). Also 
included in direct costs are training development and instructional 
materials preparation (including production) costs, or the review of 
materials purchased from a vendor. Other direct costs are materials 
and supplies, equipment, facilities, and travel and per diem. 

Organizations generally pay all employee/trainee costs. Training 
is typically conducted during working hours; consequently, trainees 
are not available to perform their regular jobs. As a result, every 





6 



6 



hour which the trainee spends in training costs the organization the 
equivalent of an hour's wages and benefits for that employee (Deming, 
1982) . 

In calculating personnel costs, wages and benefits need to be 
taken as a total compensation package. All employer-paid benefits, 
such as insurance, pensions, time paid but not worked (vacation, 
holidays, sick leave, etc.), and other contributions are included. In 
the United States, these employer-paid "fringe" benefits average 35% 
of direct salary costs (Carnevale & Schulz, 1990) . A calculation of 
total daily compensation package costs is shown below. 



EXAMPLE OF A TOTAL COMPENSATION PACKAGE 

The hourly wage for a welder is $17. In addition, the fringe 
benefits package costs the employer an additional 30%. The 
employer's cost for a welder's total daily (8-hour) 
compensation package is 



$17 


x 8 hours 


4* 


30 % 


X 


( $17 x 8 ) 


= 


$176.80 


Hourly 


Hours 




F r i nge 




Daily 




Total daily 


wage 


per day 




benef i ts 




wage 




compensation 








percent 








package 


... 


Daily wage ... ] 















Additionally, employees' time is worth more than their total 
compensation package because they are expected to contribute to the 
organization's profitability. Consequently, there can be a cost of 
disruption to productivity or a loss of productivity during training 
time. This cost becomes more apparent as the number of employees away 
from their job and the length of training increases. Nevertheless, 
the total compensation package is the generally accepted means for 
calculating trainee costs. 
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EXAMPLE OF TRAINEE COSTS 

Ten welders are attending a two-day workshop on welding 

techniques. Each welder has a total daily compensation 

package, including fringe benefits, of $176.80. The trainee 

costs associated with their participation in this workshop are 

10 x 2 days x $176.80 = $3,536.00 

Number of Length of Total daily Trainee 

trainees training compensation costs 

package 



Just as trainee costs are calculated as a direct personnel cost, 
so should instructor costs be. Yet, more may be involved than just 
the days the instructor commits to delivering training. In addition, 
preparation time should be added (Deming, 1982) . 



EXAMPLE OF INSTRUCTOR COSTS 

An instructor requires 1 day of preparation time for a two-day 
workshop. The instructor's total daily compensation package 
is $200. The instructor costs associated with this workshop 
are 



1 


X 


( 2 days 


+ 


1 day ) 


X 


$200 


= 


$600 


Number of 
instructors 




Length of 
training 




Length of 
preparation 




Total daily 
compensation 
package 




Instructor 

costs 



If an organization contracts for external training services, the 
costs of developing and delivering the training as well as the cost of 
preparing/producing instructional materials may be lumped together 
with all other costs the contractor or consultant charges. However, 
when the training is developed internally, training development and 
instructional materials preparation costs need to be calculated. 

Calculation of training development and instructional materials 
preparation costs can be made based on the time expended by the 
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developer (s) and instructional materials production personnel 
involved, and on the costs of materials and supplies required in 
preparing all types of instructional materials, including printed 
materials, audiovisual media, manipulative aids, etc. A calculation 
of development and materials preparation costs could look like the 
following example (Deming, 1982) . 



EXAMPLE OF TRAINING DEVELOPMENT AND INSTRUCTIONAL MATERIALS 
PREPARATION COSTS 

A developer spends 4 days developing a training workshop and 
an additional 7 days preparing printed instructional 
materials. No production personnel are involved. The 
developer's total daily compensation package is $225. The 
cost of materials and supplies required in preparing the 
instructional materials is $250. The training development and 
instructional materials preparation costs associated with this 
workshop are 

1 X ( 4 days + 7 days ) x $225 + $250 = $2,725 

Number of Days for Days for Total daily Cost of Development 

developers training materials compensation materials and 

development preparation package and preparation 

supplies costs 



Training development and instructional materials preparation 
costs can be treated differently than trainee and instructor costs 
insofar as the training will be repeated. For example, if the cost of 
developing the training and preparing instructional materials was 
$2,725 and the workshop was conducted 10 times, then the development 
and preparation cost assigned to any one workshop would be the total 
cost ($2,725) divided by the number of workshops (10), or $272.50 per 
workshop (Deming, 1982) . 

Another way to treat training development and instructional 
materials preparation costs is to amortize them over the number of 
trainees. For example, if the development and preparation cost was 
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$2,725 and a total of 150 welders were trained in the 10 workshops, 
then the cost per trainee would be the total cost ($2,725) divided by 
the nu mb er of trainees (150), or about $18.17 per trainee. These 
simple calculations show that (a) the more times the training is 
delivered, and (b) the more welders trained, the more economical the 
investment in training development and instructional materials 
preparation will be. 

Training managers may purchase "off-the-shelf" instructional 
materials marketed by a vendor. The purchase price of instructional 
materials such as books, modules, video tapes, etc., as well as 
consumable supplies, like pencils, paper, binders, and chalk, can be 
grouped under the heading, instructional materials costs (Deming, 
1982) . 

Some instructional materials costs are the result of multiplying 
a per-trainee cost by the number of trainees. For example, if 10 
trainees each receive a purchased $30 book, the cumulative total cost 
is $30 multiplied by 10 trainees, or $300. 



EXAMPLE OF INSTRUCTIONAL MATERIALS COSTS 

The instructional materials for a welding workshop include 
books purchased from a vendor, costing a total of $300, and 
consumable supplies which cost $75. The instructional 
materials costs associated with this workshop are 

$300 + $75 = $375 

Purchased Consumable Instructional 

instructional materials supplies materials costs 



Training involves equipment costs when machines are essential to 
effective instruction and learning. Under certain conditions, 
equipment must be rented or purchased and then maintained. In other 
instances, training can be scheduled to use equipment available in the 
organization. For example, welder training requires the use of a gas 
tungsten arc pipe welding machine. The machine's usage for training 
may, for all practical purposes, be cost-free if the training can be 
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scheduled when the machine is idle. More than likely, however, 
training workshops would be conducted during normal working hours and 
the existing pipe welding machine could not be diverted from 
production for training purposes. Consequently, a duplicate machine 
would have to be rented or purchased, and maintained. 



EXAMPLE OF RENTED EQUIPMENT COSTS 

The cost of renting a pipe welding machine for a two-day 
workshop is $150 per day. In addition, maintenance costs for 
the machine are $10 per day. The equipment costs associated 
with this workshop are 

2 days x ( $150 + $10 ) = $320 

Length of Equipment rental Maintenance Rental and 

training per day cost per day maintenance costs 

of equipment 



When equipment is purchased specifically for training, its 
purchase price can be amortized (written off) over the item's useful 
life, with yearly maintenance costs added, to find the annual cost. 

The annual cost is then distributed evenly to all training courses and 
programs in which the item is used. An example of how to calculate 
the annual cost and cost per workshop for an item of purchased 
equipment is provided on the following page. 
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EXAMPLE OP PURCHASED EQUIPMENT COSTS 

A pipe welding machine was purchased exclusively for training. 
It cost $24,000 and has an estimated useful life of 5 years. 
The machine has a yearly maintenance cost of $1,800, and it 
will be used for 10 workshops per year. The annual equipment 
cost and cost to each workshop for this machine can be 
calculated as follows. 



( $24,000 


-r 


5 years ) 


+ 


$1,800 


= 


$6,600 


Purchase price 
of equipment 




Useful life 
of equipment 




Maintenance 
cost per year 




Annua l 

equipment cost 


$6,600 


-u 


10 


= 


$660 






Annua l 

equipment cost 




Number of 
workshops 




Equipment 
cost per 
workshop 







If a workshop uses several pieces of rented or purchased equipment, 
their costs are added together (Carnevale & Schulz, 1990) . 

Facilities costs are incurred when a training facility is built, 
shared, or rented. In cases where a building or special structure is 
built for training use, the cost can be amortized over its functional 
life, with yearly maintenance costs added (as in the purchase of 
equipment) . 

Whenever training is conducted within a facility that is used for 
other organizational functions as well, the appropriate fraction of 
that facility's cost ought to be billed to training (Deming, 1982) . 
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EXAMPLE OF FACILITIES COSTS (sharing a facility) 

Organization facilities which are used 10% of the time for 
training have a yearly cost, including maintenance and 
building administration, of $18,000. The annual facilities 
costs associated with the training use of these facilities are 

$18,000 x 10 % = $1,800 

Yearly cost of facilities, Percent of time Annual facilities 

including maintenance and used for training costs for training 

building administration 



When organization facilities are used only occasionally for 
training, a daily rate method may be preferable over the percent of 
use method shown above. To calculate the daily facilities rate, the 
total annual facilities cost (for all functions) is divided by the 
number of working days in the year. If the organization in the above 
example operates 5 days per week, 52 weeks per year (260 days per 
year), then their daily facilities rate would be $18,000 divided by 
260 days, or about $69.23. The daily rate is then multiplied by the 
number of days the facility is used for training. For example, 
the facilities cost for a 2-day workshop would be $69.23 multiplied by 
2 days, or $138.46. The percent of use method may then be applied to 
the result, if appropriate. 

In those instances when the rent for a shop, classroom, seminar 
room, etc. is not a flat fee, total facilities costs for a given 
workshop are computed by multiplying the daily rental rate by the 
number of rental days. For example, the total facilities costs for a 
2 -day welding workshop held in a vocational training center that 
charges $200 per day for its welding shop would be $200 multiplied by 
2 days, or $400. 

For on-site training, travel and per diem costs are likely to be 
of little consequence. But for training away from the workplace, they 
are serious cost factors. 
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EXAMPLE OF TRAVEL AND PER DIEM COSTS 

Ten welders traveled to another city for a 2-day workshop. 
Their employer is paying the $300 airfare and $70 daily per 
diem to cover meals, lodging, and other living expenses for 
each trainee. The employer's costs for travel and per diem 
for the trainees are 

(10 x 2 days x $70 ) + ( 10 x $300 ) = $4,400 

Number of Length of Daily Number of Round-trip Travel and 

trainees training per diem trainees air fare per diem 

rate costs 



Indirect costs. Indirect costs are expenses which cannot be 
traced back to a specific training course or program, but which are 
necessary for an organization to function. Although indirect costs 
for training are less visible than direct costs, they are substantial. 

Examples of indirect costs include interest on organizational 
debt, building repairs, utilities, organizational supplies and 
equipment, administrative and staff support salaries, and expenses for 
legal, payroll, accounting, and other personnel. Organizations often 
subdivide such costs into overhead and general and administrative 
(6 & A) expenses (Carnevale & Schulz, 1990). Overhead and G & A 
expenses are generally obtained from the organization's accounting 
office because they are arrived at through allocation, sometimes on a 
judgmental basis. 

Full cost of training. The final calculation to obtain the full 
cost of a training course or program is a simple addition problem. 
First, direct costs tied to a course or program are summed, including 

1. Total compensation packages for employees involved in 
training (e.g., trainees and instructors) 

2. Fees for external training services (contractors, 
consultants, etc.) 

3. Training development and instructional materials preparation 

costs 
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4. Costs of instructional materials and consumable supplies 
purchased from a vendor 

5. Equipment costs 

6. Facilities costs 

7 . Travel and per diem costs 

Second, indirect costs are summed, e.g. , overhead and G & A costs. 
Finally, the totals from the direct and indirect costs are summed to 
obtain a grand total, the full cost of training. 



EXAMPLE OF A FULL COST OF TRAINING CALCULATION 

Total direct costs for a 2-day workshop are $10,000. Total 
indirect costs consist of allocated overhead of $1,560 and 
allocated G & A expenses of $375. The full cost of training 
is 

$ 10,000 + ( $ 1,560 + $375 ) = $ 11,935 

Total direct costs Total indirect costs Full cost of training 



Conclusion. While calculating the full cost of training is a 
first and critical step in determining cost effectiveness, monitoring 
costs is also important to planning and controlling the training 
budget. In addition, by analyzing costs, training managers are better 
able to evaluate the proportion of the organization's investment in 
specific training populations (supervisors; production, maintenance, 
or office workers; etc.), a particular program, course, or topic, and 
so forth. 

This part of the module shows that a training manager can 
calculate the costs of training, albeit with the help of those who 
have access to fiscal data. More challenging is the art of collecting 
evidence of favorable training outcomes, but it too can be done, as 
the next part of the module shows. 
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Progress Check — Part 1 — Training Costs 

Directions . Read each of the following items carefully and write in 
your answer. If you have difficulty with any of the 13 items, go back 
and review the relevant content before continuing. Use a calculator 
or another sheet of paper for your calculations. Check your answers 
against those provided in the Progress Check Feedback at the end of 
this module. 

1. Identify the greatest incentive for indicating the cost 
effectiveness of training. 



2. Name two categories of training costs. 

( 1 ) 

( 2 ) 

3. Calculate the total daily (8-hour) compensation package for an 
employee whose hourly wage is $16, and whose fringe benefits 
package costs his employer an additional 29%. 

Total daily compensation package = 

4. Calculate the training costs for 12 trainees, with individual 
total compensation packages of $195 per day, who attend a 5-day 
training program. 

Total trainee costs = 

5 . Calculate the instructor costs for a 5-day workshop with one 
instructor who requires 2 additional days for preparation. The 
instructor's direct salary is $180 per day, with a 30% fringe 
benefits package. 

Total instructor costs = 

6. Calculate the training development and instructional materials 
preparation costs for two developers, with a total daily 
compensation package cost to the organization of $220 each, who 
worked 5 days on developing training and 8 days on preparing 
instructional materials. The cost of supplies used in preparing 
the instructional materials was $210. 

Total development and preparation costs = 
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Progress Check (continued) 

7. The cost of developing training and preparing printed 
instructional materials for a 5-day seminar was $6,300. 

A. What will be the cost per seminar if it is offered once each 
month for one year? 

Cost per seminar = 

B. If the seminar is offered only twice, to a total of 30 
trainees (15 in each session) , what will be the cost per 
trainee? 

Cost per trainee = 

8. The employer purchased books and modules for each of 12 trainees, 
costing a total of $180 per trainee. Paper and pencils, which 
cost $45 total, were also required. Calculate the total cost of 
these instructional materials. 

Total instructional materials costs = 

9. Welding Associates rented equipment for a 4-day training program 
(workshop) they are conducting. The rental cost for the 
equipment is $300 per day. Maintenance costs associated with the 
equipment are $20 per day. What will be the equipment costs for 
this training program? 

Equipment costs = 

10. Welding Associates is considering purchasing the arc welding 
machine needed for training. The machine's purchase price is 
$22,500 and its estimated useful life is 5 years. In addition, 
yearly maintenance costs for the purchased machine are expected 
to be $2,100. 

A. What will be the total annual cost of the machine? 

Total annual equipment cost = 

B. If Welding Associates uses the machine in a training program 
offered 12 times per year, what will be the equipment cost 
for each program? 

Equipment cost per training program = 
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Progress Check (continued) 

11. Welding Associates estimated their total facilities costs, 
including maintenance, for the current year to be $40,000. 

A. If the facilities are used 15% of the time for training, 
what portion of the facilities costs should be allocated to 
the training budget? 

Facilities costs (for training) = 

B. If the facilities are used for training 5 days per week, 48 
weeks per year (240 days per year), what is the daily 
facilities rate for the use of these facilities? 

Daily facilities rate = 

12. Six welders will travel from their home to another city to attend 
a 3-day workshop where they will learn to use specialized 
equipment. Welding Associates agreed to pay each employee's $400 
airfare plus $75 per day for other expenses. Calculate the 
travel and per diem costs incurred by Welding Associates for this 
3 -day workshop. 

Travel and per diem costs = 

13 . Welding Associates found their total direct costs for a training 
program to be $17,000. Indirect costs, consisting of allocated 
overhead and G & A expenses, totalled $2,000. What was Welding 
Associates' full cost of training? 

Full cost of training = 
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Part 2 — Training Effectiveness 

Measuring the Effectiveness of Training 

Donald Kirkpatrick organized the measurement of training 
effectiveness (value, worth, and merit) into four levels. These 
levels are listed below, from the easiest to measure (level 1) to the 
most difficult (level 4). In general, the more levels used to measure 
a training course or program, the more complete is the evidence of its 
effectiveness . 

• Level 1 — Measurement of trainees 7 reactions to the 
training (feedback) 

• Level 2 — Measurement of knowledge and skills acquired 

• Level 3 — Measurement of trainees' use of their new 
knowledge and skills on the job 

• Level 4 — Measurement of the organization's return on the 
training investment 

Level 1 information is gathered most often with questionnaires 
handed out at the end of a course or program or sent to trainees a 
short time later. At level 2, criterion-referenced tests are used to 
measure the knowledge and skills acquired. Level 3 ascertains if 
trainees are applying the newly-acquired knowledge and skills back on 
the job. 

Level 4 determines what benefits (increased quality, 
productivity, sales, etc.) the new knowledge and skills have had on 
the organization's performance, and their worth in monetary value. At 
level 4, training managers are asking about the organization's payback 
(return) on its training investment (Gordon, 1991) . 

In most cases, it is possible and feasible to link training 
outcomes to organizational improvements. Doing so does not require 
absolute isolation of training's benefits from the possible 
contributions of other variables. Rather, it requires evidence that 
demonstrates training's valuable role (Carnevale & Schulz, 1990). 
Consequently, arguments about whether a training manager can 
absolutely separate training's influence on organizational 
improvements and isolate the impact are not pertinent. 
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Indisputable proof is difficult to come by, even when a carefully 
designed study using experimental and control groups is conducted. 
However, evidence can be collected to show that training was at least 
a major contributor to a particular operational savings or increase in 
revenue. Kirkpatrick adds that evidence is all anybody really wants, 
anyhow. "... Management isn't going to ask, 'Can you prove it?' 
They'll ask for evidence. And evidence is not all that hard to come 
by" (Gordon, 1991, p. 23). 

The key to collecting evidence of training outcomes is to 
establish a "causal link" between a specific organizational problem, 
preferably one to which monetary value can be assigned, and a 
performance deficiency. This is best done up front, before a training 
course or program is even developed. 

Rejected workpieces in a manufacturing environment provide one 
example of an organizational problem. How much does the current 
reject rate cost the organization? Are rejected workpieces the result 
of a workforce skill deficiency, as opposed to inferior materials or 
equipment malfunctions? If so, there is a causal link. 

After establishing the link between rejected workpieces and a 
skill deficiency, current reject costs are determined. The accounting 
office can provide figures for the cost of the materials used in 
manufacturing the workpiece. When this cost is added to personnel, 
equipment, and other appropriate manufacturing costs, the total cost 
of the rejected workpieces can be calculated. If the number of 
rejected workpieces declines after the workers are trained, the 
operational saving provides convincing, quantitative evidence that the 
training provided a return on the investment. 

Benefits. By selecting the outcomes (benefits) to be measured 
and linking training to those outcomes while holding, to the extent 
possible, other factors constant, level 4 measurement becomes a 
relatively simple matter. All training managers have to do is track 
the outcomes for which baseline measures were gathered before the 
training, and they will know what the payoff is. Among the most 
important outcomes (benefits) to be documented are (a) increased 
quality, productivity, sales, service, safety, and workforce 
flexibility; (b) reduced operational costs, medical insurance and 
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workers' compensation claims; and (c) lower absenteeism. Other 
outcomes, which are more difficult to document and quantify, include 
the increased stability of the workforce; improved morale, harmony, 
job satisfaction, and attitude; a lower requirement for supervision; 
the formation of selection pools for promotion; supervisory skill 
development; and improved customer relations. 

Conclusion. Once training managers learn how to calculate the 
cost and measure the effectiveness of training, they can begin to 
describe the benefits from a financial perspective. Is the training 
effort producing benefits that are greater than the costs involved? 
This, ultimately, is what higher management wants to know. 

A variety of methods are available for determining the cost 
effectiveness of training. Some are complex and cumbersome to use, 
while others are more suitable for justifying an investment in a new 
machine for a manufacturing plant, a new way of doing a job, and so 
forth. The four methods for justifying a training investment 
presented in Part 3 of this module were selected because they are (a) 
practical, (b) relatively easy to use, and (c) generally familiar to 
higher management. It must be pointed out, however, that all four 
methods have disadvantages. Consequently, none of them should be 
regarded as a precision tool. Nevertheless, these methods are based 
on accepted principles and present organized state-of-the-art 
procedures for determining the cost effectiveness of training courses 
and programs. 
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Progress Check — Part 2 — Training Effectiveness 

Directions . Read each of the following items carefully and write in 
your answer. If you have difficulty with any of the items, go back 
and review the relevant content before continuing. Check your answers 
against those provided in the Progress Check Feedback at the end of 
this module. 

1. Kirkpatrick noted that the effectiveness of training can be 

measured at four different levels. Briefly describe the levels. 

( 1 ) 



( 2 ) 



(3) 



(4) 



2. Is it possible and/or feasible to link training outcomes to 
organizational improvements without the absolute isolation of 
training's benefits from the possible contributions of other 
variables? 

Yes, it is both possible and feasible. 

It is possible, but not feasible. 

It is feasible, but not possible. 

No, it is neither possible nor feasible. 

3. What is the key to collecting evidence of training outcomes? 



4. Identify the result of selecting the outcomes to be measured and 
linking training to those outcomes. 
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Part 3 — Methods Used to Determine the Cost Effectiveness of Training 



Return on Investment (ROI) 

ROI is the rate at which training returns what was invested (its 
cost). It is an indicator of a particular course or program's value, 
worth, and merit. Of the four methods for determining the cost 
effectiveness of training presented in this module, the ROI method is 
probably the most appealing to higher management, since managers are 
accustomed to thinking in terms of return on investment. However, the 
ROI method is appropriate only when it is possible to quantify 
outcomes (benefits) in monetary units (Kearsley, 1982, p. 92). 

The following steps must be taken before a ROI report can be 
prepared: 

• Step 1 — Calculate the direct and indirect costs 
associated with the training course or program. These costs are then 
summed to obtain the full cost of training. (See Part 1 of this 
module for details.) 

• Step 2 — Gather baseline measures for those outcomes to be 
analyzed in step 3, before training occurs. This is the only way to 
know what changes took place. 

• Step 3 — Analyze the effects of training on the outcomes 
(benefits) , such as increased quality, productivity, sales, service, 
safety, and workforce flexibility; reduced operational costs, medical 
insurance and workers' compensation claims; and lower absenteeism; or 
any other measurable benefit. In order to apply the ROI method, these 
benefits must be quantified in monetary units. 

Once the full cost of training is calculated, and the outcomes 
have been analyzed, the ROI can be computed. Training managers should 
analyze only those outcomes that are accomplished, at least in part, 
by training. To create a credible ROI report, training managers must 
present evidence that is important and believable to higher 
management . 

Two common approaches for expressing the ROI for a training 
course or program are to consider operational savings and increases in 
revenue. For example, the training manager can assess whether an 
operational cost, such as accidents due to human error, is 
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significantly lower after employees master safety procedures. In this 
case, the ROI is the rate at which training costs are recovered by a 
reduction in the number or severity of accidents. 

Reductions in accidents positively affect revenue, through 
lowered health care costs, insurance premiums, and disability claims, 
and increase productivity through fewer absentee days. Higher 
management will be impressed with evidence of training's valuable role 
in achieving both operational savings and increases in revenue. 
Consequently, they will look more favorably on training as a value- 
added service instead of just a "nice to have," but dispensable, cost 
of operation. 

To calculate a ROI, total operational savings and increases in 
revenue resulting from the training are divided by the full cost of 
training. ROI expressed as a formula is 

Operational savings + increases in revenue = 

Full cost of training 

Obviously, any training for which the ROI is greater than one (1.0), 
the break-even point, is worthwhile, because the benefit derived from 
the training is greater than its cost (we are "getting out" more than 
we "put in"). However, a ROI of less than 1.0 means that the training 
investment was greater than the return. 
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EXAMPLE OF ROI 

A safety training program resulted in operational savings 
through a $35,000 reduction in accident costs/payments the 
first year. There was also an annual increase in revenue of 
$5,000 as a result of fewer absentee days. The full cost of 
training was $25,000. The program's ROI was 
$35,000 + $5,000 



1.60 ( or 160% ) 



$25,000 



This training program was a worthwhile investment. It 
returned 160% — the original $25,000 training cost plus an 
additional $15,000 (60%) in the first year. It will no doubt 
continue to provide a benefit, even without an additional 
training investment. 
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Advantages of the ROI method. 

• Provides an indicator of the training's value, worth, and 
merit. 

• Easily understood by higher management. 

• Management will be impressed with a favorable ROI report and 
will view training as a value-added service. 

Disadvantages of the ROI method. 

• It is appropriate only when it is possible to quantify 
outcomes (benefits) in monetary units. 

• Baseline measures must be gathered for outcomes before 
training occurs. 

Evidence of operational savings and increases in revenue is 
available only after the training is conducted. Therefore, its 
application in predicting a favorable return is limited. 



Note . The advantages and disadvantages of the ROI method can be 
easily compared with those of the cost-benefit ratio, bottom-line 
evaluation, and payback period methods by referring to Table 1 on 
pages 40 and 41. 
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Progress Check — Return on Investment (ROI) 

Directions . Read each of the following items carefully and write in 
your answer. If you have difficulty with any of the items, go back 
and review the relevant content before continuing. Use a calculator 
or another sheet of paper for your calculations. Check your answers 
against those provided in the Progress Check Feedback at the end of 
this module. 

1. A technical training program is saving your organization $20,000 
per year and has increased revenue by $26,000 per year. The full 
cost of training was $40,000. 

A. What was the ROI of this training investment? 

ROI = 

B. Was this a worthwhile investment? 

Yes No 

Why or why not? 



C. Would your opinion change if you could have invested the 

$40,000 in a sayings fund at a guaranteed 17% annual return 
instead? 

Yes No 

Why or why not? 
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Cost-Benefit Ratio 

The cost-benefit analysis (ratio) method is used to determine the 
ratio of the projected full cost of a given course or program to its 
predicted benefits. This method is especially suitable for justifying 
training investments when the benefits attributable to training are 
difficult to quantify in monetary units. 

The cost-benefit ratio formula is presented below: 



Projected full cost of training 
Predicted training benefits 



Cost-benefit ratio 



If the cost-benefit ratio is less than one (1.0), the training 
would be worthwhile, because its predicted benefits exceed its 
projected costs. The smaller the ratio, the stronger the 
justification for training. If the ratio is greater than 1.0, costs 
exceed benefits and the training may not be justifiable, except when 
mandated by law (compliance training) . 

The benefits of many courses and programs, such as supervisory 
skill development, are not easy to show or quantify. Benefits such as 
reduced workforce turnover, as well as improved morale, harmony, job 
satisfaction, and attitude are hard to quantify in monetary units, yet 
they should not be overlooked. No matter how difficult it may seem to 
put a value on employee turnover, for example, an effort must be made 
to quantify all benefits for this method to work. 

Technically precise and entirely objective monetary information 
simply is not available on the benefits for some "soft-skills" 
training. However, the following practical procedure provides 
appropriate figures for benefits that are difficult to quantify. 

Before training is developed and delivered, line managers are 
asked to estimate the annual operational savings they expect to result 
for their department. These managers also rate their level of 
confidence, on a 0 to 100% scale, that the training will be 
responsible for the savings. Estimated operational savings are then 
multiplied by the "confidence 11 percentage to yield a prediction of 
total cost savings (benefits) from training. 
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EXAMPLE OF COST-BENEFIT RATIO 

A company is considering supervisory skills training for its 
foremen. The enhancement of supervisory skills is expected to 
reduce workforce turnover. Line managers have been asked to 
estimate the annual operational savings they expect as a 
result of the reduced turnover and to rate their level of 
confidence that training will be responsible for the savings. 
They estimated the operational savings (from the reduced 
turnover) at $100,000, with a confidence rating of 50%. The 
full cost of the potential training program is projected to be 
$10,000. The expected cost-benefit ratio of this training 
investment is 



$100,000 x 50 % 

Because the cost-benefit ratio is less than 1.0, the training 
program is considered worthwhile. 



Although this example is brief, it illustrates the principles and 
procedures of the cost-benefit ratio method in providing evidence that 
this soft-skills course is worthwhile. 
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Advantages of the cost-benefit ratio method. 

• Especially suitable in training situations where the 
benefits are difficult to quantify in monetary units. 

• Practical procedure that provides appropriate figures for 
benefits that are difficult to quantify. 

• Shows whether training is worthwhile, before the course or 
program is developed and delivered. 

Disadvantage of the cost-benefit ratio method. 

• The procedure used to predict training benefits in monetary 
units is subjective. 



Note . The advantages and disadvantages of the cost-benefit ratio 
method can be easily compared with those of the ROI, bottom-line 
evaluation, and payback period methods by referring to Table 1 on 
pages 40 and 41. 
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Progress Check — Cost-Benefit Ratio 

Directions. Read each of the following items carefully and write in 
your answer. If you have difficulty with either item, go back and 
review the relevant content before continuing. Use a calculator or 
another sheet of paper for your calculations. Check your answers 
against those provided in the Progress Check Feedback at the end of 
this module. 

1. Mr. Whiz predicted that a computer technician training program 

for two of his employees would result in a total cost savings of 
$9,000 per year in computer repairs and down-time. He rates his 
level of confidence, that the training will be responsible for 
this savings, at 80%. The projected full cost of the technician 
training is $2,000 per person. 

A. Calculate the cost-benefit ratio for this potential training 
investment. 

Cost-benefit ratio = 



B. Do you think this training would be a worthwhile investment 
for Mr. Whiz's company? 

Yes No 

Why or why not? 
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Bottom-line Evaluation 

The bottom-line evaluation method presented here determines the 
value added by training to each trainee's productivity, and the total 
value added to the organization by the training course or program. 

The total value added to the organization is compared to the full cost 
of training to determine if the training was a worthwhile investment. 

This method incorporates two of Kirkpatrick's four levels of 
measuring the effectiveness of training. (See Part 2 of this module 
for a description of all four levels.) It collects both level 4 
(measurement of the organization's return on the training investment) 
and level 1 (measurement of trainees' reactions to the training) data 
at the same time. A questionnaire is developed to collect level 4 and 
level 1 data from trainees after they have applied, back on the job, 
what they learned in the training. 

Trainees indicate their individual opinions of (a) percent of 
job-time spent performing the task trained (T) , (b) pre-training 

productivity percentages (PI) , and (c) post-training productivity 
percentages (P2) . Along with questionnaire responses, other 
information necessary to perform a bottom-line evaluation includes a 
list of the tasks performed by the trainees and their total annual 
compensation package (S) . The bottom-line evaluation method expressed 
as a formula is 

(_s x t ) 

Total annual Percent of 

compensation job-time 

package spent performing 

task trained 

[ Annual per-task compensation ] 

The bottom-line evaluation method promotes the use of job 
analysis information in that tasks performed by the employee/trainee 
must be identified, along with percent of job-time spent on those 
tasks, in order to calculate per-task compensation (S x T, in the 
above formula) . 



( P2 - PI ) 

Percentage 
change in 
productivity 
as a result 
of training 



Value added 
by training 
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Criticisms of the bottom-line evaluation method include: (a) 

trainee questionnaire responses are subjective perceptions; and (b) 
potential for biased questionnaire input. However, the calculated 
value added can be corrected for bias using statistical methods. 

Despite the criticisms, however, this method does promote 
employee participation in decision making. Employees often feel that 
they are the best judges of their individual performance improvement 
following training. The method also appeals to management because it 
links employees' job task performance with their productivity, by 
comparing the full cost of the training with the value the 
organization receives from it. 
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EXAMPLE OF BOTTOM-LINE EVALUATION 

A training program in erecting and dismantling scaffolding was 
provided to three employees. After the employees returned to 
their job and applied the knowledge and skill acquired, they 
were asked to complete a questionnaire. Each employee 
supplied information on their perceptions of (a) percent of 
job-time spent performing the task trained, and (b) pre- and 
post-training productivity percentages. Each employee's total 
compensation package, percent of job-time spent performing the 
task trained, and pre- and post-training productivity 
percentages are presented in the table below, along with their 
individual and total value added to the organization as a 
result of the training. 



Employee/ 

trainee 


Total annual 
compensation 
package ($) 


Job- time (%) 
spent performing 
the task trained 


Component 
pay <$) 


Pre- 
training 
produc- 
tivity (%) 


Post- 
training 
produc- 
tivity (%) 


Produc- 
tivity 
gain (%) 


Value 

added 

<$) 




CS] 


IT] 


CD a SxT] 


CPI] 


CP2] 


CG » P2-P1] 


[GxD] 


1 


$29,000 


25% 


$7,250 


30% 


80% 


50% 


$3,625 


2 


$30,000 


20% 


$6,000 


40% 


90% 


50% 


$3,000 


3 


$34,000 


15% 


$5,100 


50% 


80% 


30% 


$1,530 






Total value 


added . . 






$8,155 



The total value added to the organization by the training 
program was $8,155. If the full cost of training was less 
than $8,155, then the training program was a worthwhile 
investment . 
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Advantages of the bottom-line evaluation method. 

• Promotes the use of job analysis information. 

• Promotes employee participation in decision-making. 

• Appeals to management, because it links job task performance 
with training and productivity. 

• Allows the total value added to the organization to be 
compared to the full cost of training to determine if the training was 
a worthwhile investment. 

Disadvantages of the bottom-line evaluation method. 

• Questionnaire responses are subjective perceptions by 
trainees . 

• Questionnaire data might be biased. 



Note . The advantages and disadvantages of the bottom-line 
evaluation method can be easily compared with those of the ROI, 
cost-benefit ratio, and payback period methods by consulting 
Table 1 on pages 40 and 41. 
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Progress Check — Bottom-Line Evaluation 

Directions. Read each of the following items carefully and write in 
your answer. If you have difficulty with any of the items, go back 
and review the relevant content before continuing. Use a calculator 
or another sheet of paper for your calculations. Check your answers 
against those provided in the Progress Check Feedback at the end of 
this module. 



1. A training manager has identified all the tasks performed by each 
employee within the organization, along with their total 
compensation packages. The training department is providing a 
welding training program for the welding department's four 
employees, at a total cost of $4,400. Each welder has a total 
annual compensation package of $20,000. The training manager 
would like to present training's value to higher management. The 
training is delivered and questionnaire results, in table form, 
are as follows. 



Employee/ 

trainee 


Total annual 
compensation 
package ($) 


Job- time (%) 
spent performing 
the task trained 


Component 
pay ($) 


Pre- 

training 
produc- 
tivity (%) 


Post- 
training 
produc- 
tivity (%) 


Produc- 
tivity 
gain (%) 


Value 

added 

($) 




[SI 


CT3 


CD = SxT] 


CPI] 


CP23 


CG = P2-P13 


[GxD] 


i 




10% 




20% 


60% 






2 




40% 




20% 


50% 






3 




30% 




30% 


50% 






4 




20% 




40% 


60% 










Total value 


added . . 









A. Complete the table. For each employee, you will need to 

calculate component pay, productivity gain, and value added. You 
will also need to calculate the training program's total value 
added to the organization. 
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Progress Check (continued) 

B. Was the training program worthwhile? 

Yes No 

Why or why not? 
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Payback Period 

The fourth method of determining the cost effectiveness of a 
training investment is called the payback period method. This 
forecasting method answers the guestion, "How long will it take the 
training to pay for itself?" This method should, however, be used 
only as an initial look at a questionable training investment. 

The payback period method does not consider the cost or time 
value of the money spent and tied up before, during, and after the 
training until the break-even point is reached. Nevertheless, it does 
consider some time factors in calculating the payback. 

If the payback period is very short, less than one year, for 
example, then the training course or program is definitely promising 
and another method, such as ROI, cost-benefit ratio, or bottom-line 
evaluation, should be used for a closer examination of the training's 
value, worth, and merit to the organization. If the payback period is 
very long, 10 years, for example, then there may be no need to 
consider the training further, depending on the organization's 
philosophy about the maximum length of time allowed for investment 
resources to be returned. 

The payback period method is represented by the following 
formula: 

Full cost of training = Payback period (in years) 

Annual operational savings + increase in revenue 
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EXAMPLE OF PAYBACK PERIOD 

A company is considering an all-inclusive clerical training 
program. The full cost of this training is $10,000. Company 
managers estimate that improved clerical effectiveness 
resulting from the training will save $1,200 per month 
($14,400 per year). Should the company give further 
consideration to this training? 

= 0.69 years (about 8.3 months or 36 weeks) 

$14,400 

Yes, the company should consider the training further. Its 
payback period is very short, less than one year. Further 
calculation shows that, if the company's savings estimates are 
accurate, this investment (a) has a worthwhile cost-benefit 
ratio of only 0.69, and (b) will return 144% in the first year 
(the training year) . 
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Advantages of the payback period method* 

• Provides a quick initial look at a potential training 
investment . 

• Answers the question, "How long will it take the training to 
pay for itself?" 

Disadvantages of the payback period method* 

• Should be used only as a screening tool. If the payback 
period is short, then another method (ROI, cost-benefit ratio, or 
bottom-line evaluation) must be used to examine the training's value, 
worth, and merit to the organization. 

• Does not consider the cost or time value of the money spent 
and tied up before, during, and after the training until the break- 
even point is reached. 



Note . The advantages and disadvantages of the payback period 
method can be easily compared with those of the ROI, cost-benefit 
ratio, and bottom-line evaluation methods by consulting Table 1 
on the following pages. 
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Table 1 

Advantages and Disadvantages of Methods for Determining 
the Cost Effectiveness of Training 



Advantages 



Disadvantages 



Return on investment (ROI) 



Provides an indicator of 
training's value, worth, and 
merit. 

Easily understood by higher 
management . 

Management will be impressed with 
favorable ROI report and view 
training as a value-added 
service. 



Appropriate only when it is 
possible to quantify outcomes 
(benefits) in monetary units. 

Baseline measures must be 
gathered for outcomes before 
training occurs. 

Evidence of operational savings 
and increases in revenue 
available only after training is 
conducted. Therefore, ROI's 
application in predicting a 
favorable return is limited. 



Cost-benefit ratio 

Especially suitable in situations Procedure used to predict 
where the benefits are difficult benefits in monetary units is 
to quantify in monetary units. subjective. 

Practical procedure that provides 
appropriate figures for benefits 
that are difficult to quantify. 

Shows whether training is worth- 
while, before course or program 
is developed and delivered. 



( Table continues ) 
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Table 1. 



(continued) 



Advantages 


Disadvantages 


Bottom-line 


evaluation 


Promotes use of job analysis 


Questionnaire responses are 


information. 


subjective perceptions by 
trainees . 


Promotes employee participation 
in decision-making. 


Questionnaire data might be 
biased. 


Appeals to management, because it 
links job task performance with 
training and productivity. 




Total value added to the 
organization can be compared to 
full cost of training to 
determine if training was a 
worthwhile investment. 





Payback period 



Provides quick initial look at 
potential training investment. 

Answers question, "How long will 
it take training to pay for 
itself?" 



Should be used only as screening 
tool. If payback period is 
short, then another method must 
be used to examine training's 
value, worth, and merit to 
organization . 

Does not consider cost or time 
value of money spent and tied up 
before, during, and after 
training until the break-even 
point is reached. 
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Progress Check — Payback Period 

Directions . Read each of the following items carefully and write in 
your answer. If you have difficulty with either item, go back and 
review the relevant content before continuing. Use a calculator or 
another sheet of paper for your calculations. Check your answers 
against those provided in the Progress Check Feedback at the end of 
this module. 

1. Fixit Company is considering a training workshop for 12 

employees. The full cost of the training workshop (for all 12 
employees) is $9,000. The training manager has estimated that 
the additional training would enable the company to increase its 
revenue by $5,000 per year. He has decided that if the payback 
period was no more than 2 years, then he would give further 
consideration to the workshop. 

A. Calculate the payback period for this potential training 
investment. 

Payback period = 

B. Should the training manager consider the training workshop 
further? 

Yes No 

Why or why not? 
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Practical Example skill Check 



Background information. The training manager for the Camel 
Company had no interest in or idea how to determine the cost 
effectiveness of training. Consequently, the use of economic 
justification was avoided, and higher management came to view training 
as a cost of doing business. During an economic downturn, most of the 
training staff was cut as a result of "down-sizing." As a result, the 
training manager decided that he had better learn how to justify 
training as a worthwhile investment if he wanted to keep his job. 

The training manager's first effort was to establish a "causal 
link" between a specific organizational problem and a performance 
deficiency. Finally, he had to provide evidence, in monetary units, 
that training was a worthwhile investment for the company. 

Because the general manager of the Camel Company was accustomed 
to making decisions based on return on investment (ROI) reports, the 
training manager chose this method to justify the cost effectiveness 
of training. 

Answers to the following items are provided in the Practical 
Example Skill Check Feedback at the end of this module. 

I. List the three steps the training manager must take before 

preparing a return on investment (ROI) report for the general 
manager? 

( 1 ) 



( 2 ) 



(3) 
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Practical Example Skill Check (continued) 

II. The following is a list of outcomes that could be measured. To 
prepare a credible ROI report, the evidence presented to the 
general manager should be described in which of the following 
terms? Place an X in front of those outcomes (benefits) which 
would provide convincing evidence that training was a worthwhile 
investment. 

1. How those trained thought the training program went 

2. Any increase in operational savings attributable to the 

training 

3. Test scores showing that all trainees passed the post- 
test (final exam) 

4. How efficient the trainees' supervisors thought the 

training program was 

5 . increased revenue attributable to enhanced proficiency 

of the workers after training 

6. Suggestions for improving/deleting parts of the 

training, derived from the training exit survey 

7. Reduced medical claims attributable to safety awareness 

from the training program 

8. Company executives' feelings about the training program 

9. Reduced operational costs 
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Practical Example skill Check (continued) 

III. List the advantages and disadvantages of two other methods that 
the training manager could use to determine the cost 
effectiveness of training. 

Method Advantages Disadvantages 
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Practical Example Skill Check (continued) 

IV. Ten employees in the maintenance department went through a one- 
week (40-hour) training program. The training costs were as 



follows: 

Cost item Amount 

Workers' total weekly compensation 

package (10 workers) $ 8,500 

Trainers' total weekly compensation 

package (2 trainers) $ 4,200 

Training development and instructional 

materials preparation $ 1,000 

Purchased instructional materials and 

consumable supplies $ 450 

Equipment rental $ 1,000 

Allocated facilities costs 

($70 per day) $ 350 

Overhead and G & A expenses (total) $ 500 

Full cost of training $16,000 



Actual benefits of the program were difficult to quantify. 
However, the benefit item amounts listed on the next page were 
found by (a) taking figures for each item from the 12 monthly 
periods before the training; (b) then obtaining an average of the 
12 figures for each item reported; (c) calculating each item's 
average monthly benefit (the difference between the item average 
and the figure for the same item after training) ; and (d) 
annualizing the resulting monthly benefit (by multiplying by 12) . 
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Practical Example skill Check (continued) 



IV. (continued) 

Benefit item 



Annualized benefit amount 
from training 



Operational costs (reduced) $ 9,600 

Productivity (increased) $13,200 

Absenteeism (decreased) $ 3,000 

Quality (improved) $ 2,400 

Total annual benefit $28,200 



After applying, back on the job, what they learned in training, 
the employee/trainees completed a questionnaire. The response 
data indicated that, on average, 

(1) 35% of their job-time was spent performing the task trained 

(2) Their productivity in performing the task trained improved 
28% as a direct result of the training. 

Using the information given (in item IV) , and the payback period 
method, how long before the break-even point for this training 
was reached? Show all calculations. 

Payback period = 
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Practical Fyample Skill Check (continued) 



V. Continuing with the information from item IV, and using the ROI, 
cost-benefit ratio, and bottom-line evaluation methods, determine 
if this training should be continued or terminated. Describe 
your rationale. Use the following criteria: "Any activity in 

which the organization is involved, including any department 
within the organization, should add appreciably to our goal of 
11.5% increase in total annual revenue over the next five years." 
(from Camel Company's Vision. Values, and Goals statement.) 

ROI = 

Cost-benefit ratio (using calculated costs and benefits) = 



Bottom-line evaluation (total value added) = 



Determination : 

Continue the training 
Terminate the training 



Rationale: 
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Progress Check Feedback — 

Part 1 — Training Costs (pp. 15-17) 

1. Identify the greatest incentive for indicating the cost 
effectiveness of training. 

JjistJJ&inci trainijig_ eicgendrturejs with documented 

benefits (jg_. 4J 

2. Name two categories of training costs. 

(1) direct costjs 

(2) indirect costjs jg_. 5) 

3. Calculate the total daily (8-hour) compensation package for an 
employee whose hourly wage is $16, and whose fringe benefits 
package costs his employer an additional 29%. 

Total daily compensation package = 

(Jjl_6 x 8 hoursj ± [29% x $16 x 8 hours ) = $165.12 

4. Calculate the training costs for 12 trainees, with individual 
total compensation packages of $195 per day, who attend a 5-day 
training program. 

Total trainee costs = 12 trainees x 5 days. K §J_9_5 ~ §_lA-e_Z00 

5. Calculate the instructor costs for a 5-day workshop with one 
instructor who requires 2 additional days for preparation. The 
instructor's direct salary is $180 per day, with a 30% fringe 
benefits package. 

Total instructor costs = 

1 instructor, x__[5_jt_2 da^sj x j§JJW ± ($180 x 30%±J ~ 

6. Calculate the training development and instructional materials 
preparation costs for two developers, with a total daily 
compensation package cost to the organization of $220 each, who 
worked 5 days on developing training and 8 days on preparing 
instructional materials. The cost of supplies used in preparing 
the instructional materials was $210. 

Total development and preparation costs = 
j2 developers x (_8 ± 5 days} x $220 7 + $210 = $5.920 
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Progress Check Feedback — Part 1 — Training Costs (continued) 



7 . The cost of developing training and preparing printed 
instructional materials for a 5-day seminar was $6,300. 

A. What will be the cost per seminar if it is offered once each 
month for one year? 

Cost per seminar = $6,300 ± 12 times offered = 5525 

B. If the seminar is offered only twice, to a total of 30 
trainees (15 in each session) , what will be the cost per 
trainee? 

Cost per trainee = $6,300 ± 30 trainees^ = $210 

8. The employer purchased books and modules for each of 12 trainees, 
costing a total of $180 per trainee. Paper and pencils, which 
cost $45 total, were also required. Calculate the total cost of 
these instructional materials. 

Total instructional materials costs = 

L§_180 x 12 trainees ) + $45 = $2,205 

9. Welding Associates rented equipment for a 4-day training program 
they are conducting. The rental cost for the equipment is $300 
per day. Maintenance costs associated with the equipment are $20 
per day. What will be the equipment costs for this training 
program? 

Equipment costs = 4 days x (_$300 ± $20) = $1,280 

10. Welding Associates is considering purchasing the arc welding 
machine needed for training. The machine's purchase price is 
$22,500 and its estimated useful life is 5 years. In addition, 
yearly maintenance costs for the purchased machine are expected 
to be $2,100. 

A. What will be the total annual cost of the machine? 

Total annual equipment cost = 

f$22,500 + 5 years) + $2,100 = $6,600 

B. If Welding Associates uses the machine in a training program 
offered 12 times per year, what will be the equipment cost 
for each program? 

. Equipment cost per training program = 

$6,600 annual cost ± 12 times offered = $550 
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Progress Check Feedback — Part 1 — Training Costs (continued) 

11. Welding Associates estimated their total facilities costs, 
including maintenance, for the current year to be $40,000. 

A. If the facilities are used 15% of the time for training, 
what portion of the facilities costs should be allocated to 
the training budget? 

Facilities costs (for training) = $40,000 x 15% = $5,000 

B. If the facilities are used for training 5 days per week, 48 
weeks per year L5 days x 48 weeks = 240 dags per year? . what 
is the daily facilities rate for the use of these 
facilities? 

Daily facilities rate = $40,000 + 240. dags = $155.57 

12. Six welders will travel from their home to another city to attend 
a 3-day workshop where they will learn to use specialized 
equipment. Welding Associates agreed to pay each employee's $400 
airfare plus $75 per day for other expenses. Calculate the 
travel and per diem costs incurred by Welding Associates for this 
3 -day workshop. 

Travel and per diem costs = 

(_6 welders x 3 da^s x $75) + (_6 x $ 400 air fare) = $3,750 

13. Welding Associates found their total direct costs for a training 
program to be $17,000. Indirect costs, consisting of allocated 
overhead and G & A expenses, totalled $2,000. What was Welding 
Associates' full cost of training? 

Full cost of training = $17,000 + $2,000 = $13,000 
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Progress Check Feedback — 

Part 2 — Training Effectiveness (p. 21) 

1. Kirkpatrick noted that the effectiveness of training can be 

measured at four different levels. Briefly describe the levels. 
(1) [See page 18J 



( 2 ) 



(3) 



(4) 



2. Is it possible and/or feasible to link training outcomes to 
organizational improvements without the absolute isolation of 
training's benefits from the possible contributions of other 
variables? 

X Yes, it is both possible and feasible. j 'P . 18J 

It is possible, but not feasible. 

It is feasible, but not possible. 

No, it is neither possible nor feasible. 

3. What is the key to collecting evidence of training outcomes? 

Establish a causal link between a specific organizational 

problem, preferably a problem to which monetary value can be 
assigned , and a performance deficiency_. (_g_. 19_j 

4. Identify the result of selecting the outcomes to be measured and 
linking training to those outcomes. 

Level 4 measurement becomes, a relatively simgle matte r. [g. 19J 
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Progress Check Feedback — 

Return on Investment (ROI) (p. 26) 

1. A technical training program is saving your organization $20,000 
per year and has increased revenue by $26,000 per year. The full 
cost of training was $40,000. 

A. What was the ROI of this training investment? 

ROI = j $20. , 000 ± $26,000) ± $40^000 = 1.15 or 115% 

B. Was this a worthwhile investment? 

Yes X No 

Why or why not? 

The ROI is greater than one (1.0), the break-even goint_. 

The training investment is returning 115% per year. The 
first year, this is the original training investment plus 
an additional $6,000 (or 15%). The training will no doubt 

continue to provide a benefit , even without additional 

investment . 

C. Would your opinion change if you could have invested the 
$40,000 in a savings fund at a guaranteed 17% annual return 
instead? 

Yes No X 

Why or why not? 

In the first year, the 17 % return on savings would be 

better, than the 15% return from the trarninq. iiivestment_. 
However . , the trerning. investment will no doubt cgntrnue to 
provide a return without an additional investment . 
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Progress Check Feedback — 

Cost-Benefit Ratio (p. 30) 

1. Mr. Whiz predicted that a computer technician training program 

for two of his employees would result in a total cost savings of 
$9,000 per year in computer repairs and down-time. He rates his 
level of confidence, that the training will be responsible for 
this savings, at 80%. The projected full cost of the technician 
training is $2,000 per person. 

A. Calculate the cost-benefit ratio for this potential training 
investment. 

Cost-benefit ratio = 

(_2 trainees x $2.000) ± f$9.000 x 80%) = 0.555 

B. Do you think this training would be a worthwhile investment 
for Mr. Whiz's company? 

Yes X No 

Why or why not? 

The cost^benefit ratio of this potential twining 

iMvestment is less than one (1.0). ( P . 27J 
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Progress Check Feedback — 

Bottom-Line Evaluation (pp. 35-36) 

1. A training manager has identified all the tasks performed by each 
employee within the organization, along with their total 
compensation packages. The training department is providing a 
welding training program for the welding department's four 
employees, at a total cost of $4,400. Each welder has a total 
annual compensation package of $20,000. The training manager 
would like to present training's value to higher management. The 
training is delivered and questionnaire results, in table form, 
are as follows. 



Employee/ 

trainee 


Total annual 
compensation 
package ($) 


Job-time (%) 
spent performing 
the task trained 


Component 
pay ($) 


Pre- 

training 
produc- 
tivity (%) 


Post- 
training 
produc- 
tivity (%) 


Produc- 
tivity 
gain (%) 


Value 

added 

($) 




csi 


[T3 


[D = SxT] 


[PI] 


CP2] 


[G = P2-P1] 


[GxD] 


1 


$ 20,000 


10% 


$ 2,000 


20% 


60% 


40 % 


$ 800 


2 


$ 20,000 


40% 


$ 8,000 


20% 


50% 


30 % 


$ 2,400 


3 


$ 20,000 


30% 


$ 6,000 


30% 


50% 


20 % 


$ 1,200 


4 


$ 20,000 


20% 


$ 4,000 


40% 


60% 


20 % 


$ 800 






Total value 


added . . 






$ 5,200 



A. Complete the table. For each employee, you will need to 
calculate component pay, productivity gain, and value added. You 
will also need to calculate the training program's total value 
added to the organization. 

B. Was the training program worthwhile? 

Yes X No 

Why or why not? 

The value added to the orgariiza±ion by the t^rajjiing^ /$5,200) 

was greater than the training, investment (_§4_ l _400±_. 
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Progress Check Feedback — 

Payback Period (p. 42) 



57 



1. Fixit Company is considering a training workshop for 12 

employees. The full cost of the training workshop (for all 12 
employees) is $9,000. The training manager has estimated that 
the additional training would enable the company to increase its 
revenue by $5,000 per year. He has decided that if the payback 
period was no more than 2 years, then he would give further 
consideration to the workshop. 

A. Calculate the payback period for this potential training 
investment. 

Payback period = §9_ t 000_± $5.000 = 1.8 years 

B. Should the training manager consider the training workshop 
further? 

Yes X No 

Why or why not? 

The payback geriod was less, than the t^raining_ manager's 
cut-off, of. 2 Years.:. 
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Practical F-y ample Skill Check Feedback (pp. 43-48) 



I. 



II, 



List the three steps the training manager must take before 
preparing a return on investment (ROI) report for the general 
manager? 

( 1 ) Calculate the direct ajjd indirect costs; associated with 



the tyrajjiijiCL. 
traJjlinQ . 



Then add these costs to obtain the full cost of 



(2) Baseline (before training) those outcomes to be 



analyzed in step, 3_. 



(3) Analyze the effects. of_ t^aining_ on the outcomes 



(JaenefitsJ, in monetary. units_. [£_. 22J 



The following is a list of outcomes that could be measured. To 
prepare a credible ROI report, the evidence presented to the 
general manager should be described in which of the following 
terms? Place an X in front of those outcomes (benefits) which 
would provide convincing evidence that training was a worthwhile 
investment. 

1. How those trained thought the training program went 

X 2. Any increase in operational savings attributable to the 

training (pp. 22-23) 

3 . Test scores showing that all trainees passed the final 

exam 

4. How efficient the trainees' supervisors thought the 

training program was 

X 5. Increased revenue attributable to enhanced proficiency 

of the workers after training (pp. 22-23) 



6. Suggestions for improving/deleting parts of the 
training, derived from the training exit survey 

7. Reduced medical claims attributable to safety awareness 
from the training program (P. 22J 

8. Company executives' feelings about the training program 

9 . Reduced operational costs (pp. 22-23) 
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Practical Example Skill Check Feedback (continued) 

III. List the advantages and disadvantages of two other methods that 
the training manager could use to determine the cost 
effectiveness of training. 

Method Advantages Disadvantages 

(See Table 1, pages 40 and 41) 
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Practical Exa mple s kill Check Feedback ( cont i nued ) 

IV. Ten employees in the maintenance department went through a one- 
week (40-hour) training program. The training costs were as 



follows: 

Cost item Amount 

Workers 7 total weekly compensation 

package (10 workers) $ 8,500 

Trainers 7 total weekly compensation 

package (2 trainers) $ 4,200 

Training development and instructional 

materials preparation $ 1,000 

Purchased instructional materials and 

consumable supplies $ 450 

Equipment rental $ 1,000 

Allocated facilities costs 

($70 per day) $ 350 

Overhead and G & A expenses (total) $ 500 

Full cost of training $16,000 



Actual benefits of the program were difficult to quantify. 
However, the benefit item amounts listed on the next page were 
found by (a) taking figures for each item from the 12 monthly 
periods before the training; (b) then obtaining an average of the 
12 figures for each item reported; (c) calculating each item 7 s 
average monthly benefit (the difference between the item average 
and the figure for the same item after training) ; and (d) 
annualizing the resulting monthly benefit (by multiplying by 12) . 
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Practical Example Skill check Feedback (continued) 
IV. (continued) 



Benefit item 


Annualized benefit amount 
from training 


Operational costs (reduced) 


$ 9,600 


Productivity (increased) 


$13,200 


Absenteeism (decreased) 


$ 3,000 


Quality (improved) 


$ 2.400 



Total annual benefit $28,200 



After applying, back on the job, what they learned in training, 
the employee/trainees completed a questionnaire. The response 
data indicated that, on average, 

(1) 35% of their job-time was spent performing the task trained 

(2) Their productivity in performing the task trained improved 
28% as a direct result of the training. 

Using the information given (in item IV), and the payback period 
method, how long before the break-even point for this training 
was reached? Show all calculations. 

Payback period = $16.000 ± $ 2^200 = 0.57 years 

[about 6.8 months or 29_±5 weeks ) 
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Practical Example Skill Check Feedback (continued) 

V. Continuing with the information from item IV, and using the ROI, 
cost-benefit ratio, and bottom-line evaluation methods, determine 
if this training should be continued or terminated. Describe 
your rationale. Use the following criteria: "Any activity in 

which the organization is involved, including any department 
within the organization, should add appreciably to our goal of 
11.5% increase in total annual revenue over the next five years." 
(from Camel Company's Vision , Values , and Goals statement.) 

ROI = 525,200 ± $16^000 = 1.76 or 176% 

Cost-benefit ratio (using calculated costs and benefits) = 

516 , 000 ± j28_ L 2M = 0.57 

Bottom-line evaluation (total value added) = 

/•/'68.5Q0 x 52 weeks ) x 35% job-time on taskj x 28 % 

improvement = 543,516 

Determination : 

Continue the training X 

Terminate the training 

Rationale: 

* The training, returns the investment plus an additional 

76% in the first far more than the stipulated 11^5% 

annual return. 

« The costjzbenefit ratio is well below one (1.0). 

« The value added to the organization b}r the training is more 
than 2 times greater than the training investment . 
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AN ACCOUNTING FRAMEWORK FOR INVESTMENTS IN HUMAN RESOURCE DEVELOPMENT 
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Individuals Governments Employers 
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Direct costs, if any Acquisition of None (private Enhanced social None Improved 

knowledge, skills, funds) to a full cohesion + knowledge & skills 

& qualifications subsidy (public advanced of workforce 

leading to a better funds) knowledge & skills 



Page 3 of 3 

'Formal education is a system with a hierarchic structure and chronological succession of grades, from kindergarten through the 
university which, in addition to academic studies, comprises a variety of specialized programs and full-time training institutions. 
Classes are conducted within specially built schools, centers, institutes, or colleges. Strictly controlled credentials form the basis of 
legitimacy. 
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Human Resource Development Department 501 
Survey of Human Resource Development 



The University of Tennessee 
Progress test No. 1 1 



LEGAL ASPECTS OF TRAINING 



Directions: Each of the following items contains four (4) alternatives, one of which provides the 

best answer. Circle the letter which precedes the best answer. 



1 . It is important to determine whether training time 

is compensable under federal legislation because 

A. training cannot be conducted during regular 
work hours. 

B. all participants in training would have to be 
paid overtime. 

C. participants in some training programs would 
have to be paid overtime. 

D. salaried employees taking part in training 
would have to be paid overtime. 

2. Under federal wage and hour laws, 

A. all trainees and apprentices are exempt from 
minimum wage requirements. 

B. trainees and apprentices can never be paid 
subminimal rates. 

C. trainees and apprentices may be paid 
subminimal rates, under certain defined 
circumstances. 

D. trainees and apprentices are considered the 
same as employees who take college courses 
at night. 

3. Which of the following would most likely qualify 

as true apprenticeship training eligible for U.S. 

government guidance and assistance? 

A. A one-year work/study program to train 
clerk-typists. 

B. An on-the-job sales training program with no 
classroom instruction. 

C. A work experience training program for 14- 
to 16-year-old migrants to learn harvesting 
techniques. 

D. A 5-year program combining classroom 
instruction and on-the-job experience to train 
electricians. 



20 
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Human Resource Development Department The University of Tennessee 

Training for Human Resource Development Job Performance Aid 



CHECKLIST FOR PRESENTATION SKILLS 



Answer "yes" or "no" to all questions. 

1 . Did you prepare an introduction which includes a statement of the purpose of your 

presentation? 

2. Does the introduction orient learners to the subject matter of your presentation? 

3. Have you chosen a presentation format or instructional strategy appropriate for your 

learners' education, experience, and level of comprehension? 

4. Is the type of presentation (lecture, demonstration, role play, etc.) right for the size of 

your group and the subject matter? 

5. Are you comfortable with this kind of presentation? Whether formal or informal, does 

it fit your natural delivery style? 

6. Is the nature and amount of your material suited to learners' levels of education, 

experience, and comprehension? 

7. Is the material relevant to learners? 

8. During your presentation, do you ascertain whether learners are listening to and 

understanding your message? To make your determination, do you note verbal and 
nonverbal (gestures, facial expressions, etc.) cues from learners? 

9. Are these cues also useful in determining the length, focus, pace, activities, and 

illustrations of the presentation? 

10. During the presentation, do you provide visual or verbal illustrations of the main 

points? 

11. When you use analogies, do you select them to illustrate or clarify a particular point? 

Do you select analogies to compare known and unknown variables or two similar 
entities? Do your analogies fit logically into the context of your presentation? Are they 
well matched to learners' levels of comprehension and experience? 
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12. If you establish frames of reference during the presentation, do they relate directly to a 
point you are making? Are they instrumental in relating the subject matter to the 
learners? Are they well matched to learners' levels of comprehension and experience? 

13. At the end of the presentation, do you provide summaries of key points? 

14. Do you apply feedback from learners to evaluate the presentation? 

15. Do you speak clearly and loudly enough for the group to hear everything you say? Is 
your tone of voice relaxed and natural? Are your mannerisms and expressions engaging 
rather than distracting? Do you maintain eye contact with your learners? 

16. Are you enthusiastic about the subject of your presentation and do you project this 
excitement? Do you also communicate the credibility of your subject? 

17. Do you show learners that you respect their background and previous experience? 



Note . Adapted from How to Create a Good Learning Environment by M. Callahan (Ed.), INFO- 
LINE, Number 506, 1985, Alexandria, Virginia: American Society for Training and 
Development. 



O 

ERIC 




c 2 



299 



Page 1 of 7 



Human Resource Development Department The University of Tennessee 

Survey of Human Resource Development Final Exam (post-test) 



NAME: S.S.# 

Directions: Each of the following items contains four (4) alternatives, one of which provides the 

best answer. Circle the letter which precedes the best answer. 

1. A result of America's change from an agrarian to an industrial economy was 

A. reduced interest in training programs because of high unemployment. 

B. the substitution of apprenticeship for most other types of training. 

C. a de-emphasis on agricultural training. 

D. the establishment of vocational education in schools to augment apprentice training. 

2. History shows that the greatest stimulus to the growth of training in the United States has been 

A. wartime skilled workforce needs. 

B. the prosperity of the 1920' s. 

C. the decreasing influence of the profit motive. 

D. skilled worker supply /demand imbalances. 

3. The program to train shipyard workers during World War I was based on the principle of 

A. show, tell, do and check. 

B. classroom training separated from the worksite. 

C. traditional apprenticeship training. 

D. avoiding the use of work supervisors as trainers. 

4. It is important to determine whether time spent in training is compensable time under 
federal legislation because 

A. participants in some training programs would have to be paid overtime and training 
costs would be higher. 

B. salaried employees taking part in training would have to be paid overtime. 

C. training can not be conducted during regular work hours. 

D. all participants in training would have to be paid overtime. 

5. Which of the following would most likely qualify as true apprenticeship training eligible for 
U.S. government guidance and assistance? 

A. A one-year work/study program to train clerk-typists. 

B. An on-the-job sales training program with no classroom instruction. 

C. A 5-year program combining classroom instruction and on-the-job experience to train 
electricians. 

D. A work experience training program for 14- to 16-year-old migrants to learn harvesting 
techniques. 
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6. The Equal Employment Opportunity Commission's guidelines for selecting employees 

A. prohibit the use of selection tests. 

B. view all pencil-and-paper tests as being fairer than other selection methods. 

C. do not provide guidance in the use of employment selection tests. 

D. allow the use of valid selection tests 

7. Financial benefits are typically easiest to measure in programs to train 

A. instructors. 

B. first-line supervisors. 

C. production skills. 

D. management practices. 

8. A recommended procedure for achieving cost-effective training is to 

A. concentrate on training costs since benefits cannot be measured in financial terms. 

B. determine the absolute minimum training needs and deliver low-cost programs to meet 
these. 

C. always use the lowest-cost training method available. 

D. design and estimate costs for an optimum program and compare alternatives with this. 

9. The package presented by the training department to support its operating and capital 
budgets usually should include 

A. training needs, programs, costs, and benefits. 

B. information on scheduling and participant costs. 

C. no mention of objectives other than financial ones. 

D. substantial excess costs to allow for cutting. 

10. What is the advantage of charging the costs of a training program back to the using 
department or organization? 

A. The training department gains control of some of the operating department's funds. 

B. It automatically establishes the financial benefits of the training program. 

C. It substantially reduces the total cost of training to the overall organization. 

D. It reduces the training department budget by transferring some costs to the users. 

11. The Hawthorne studies from 1927-32 found that 

A. higher wages are the best incentive for increased productivity. 

B. physical and environmental influences have little effect on worker efficiency. 

C. social interactions and group influences may have a stronger effect on work efficiency than 
physical factors in the environment. 

D. experimental methods are of little value in improving work effectiveness. 
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12. Who is regarded as the "father of scientific management"? 

A. Elton Mayo 

B. W. Edwards Deming 

C. Frederick Taylor 

D. Samuel Gompers 

13. One argument in favor of Douglas McGregor's Theory Y philosophy is that 

A. Theory Y allows a range of management approaches; Theory X permits only one. 

B. employees will always seek responsibility. 

C. achieving security is the main motivation of employees. 

D. subordinates will always use self-direction in meeting organizational goals. 

14. Maslow's hierarchy of human needs predict that a worker whose physiological, safety, social, 
and ego needs were satisfied would be motivated by 

A. salary increases. 

B. chances to satisfy self-fulfillment needs. 

C. chances to satisfy belonging and acceptance needs. 

D. fear of punishment or loss of reward. 



15. Managers can conclude that a formal training program will be useful when 

A. specific skills are required for successful job performance. 

B. problems exist that can be solved by giving personnel identifiable items of knowledge, 
skill, or attitude. 

C. individuals’ performance does not meet job standards. 

D. the organization is not meeting goals included in annual or long-term plans. 

16. When originally organizing training efforts, the first task is to 

A. decide on the internal organization of the training department. 

B. survey the training that competitors use. 

C. recruit staff for the training department. 

D. study the organization to determine training needs. 

17. To perform the evaluation function successfully, 

A. a comprehensive evaluation must be conducted. 

B. operating plans must be as long term as possible. 

C. courses and programs must have performance objectives. 

D. training activities and staff must be organized by function. 

18. The most important function of a training policy is to 
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C. show how training will help achieve organizational (workplace) goals. 

D. convince top management that training is necessary. 
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19. When an organization selects internal candidates to join the training staff, managers should 

A. invest the resources needed for new trainers to develop essential job skills. 

B. choose those who need no further training to perform their new duties. 

C. make sure first that no qualified people are available from outside the organization. 

D. select only those who are subject matter experts with long experience. 

20. The first step in filling a training position is to 

A. publicize the vacancy. 

B. contact professional organizations. 

C. specify the job requirements. 

D. inform employment agencies of the opening. 

21. When a candidate selected for a training position is deficient in knowledge, skills, or other 
selection criteria, the best thing for the training manager to do is 

A. redefine the job specifications. 

B. change the selection criteria for future positions. 

C. invest in the necessary training and development. 

D. reduce the job performance requirements. 

22. Training and development needs result from 

A. future responsibilities requiring new skills. 

B. organizational goals of helping employees achieve their potential. 

C. current human performance not being what is required by organizational objectives. 

D. all of the above. 

23. A customer survey may be carried out when investigating training needs because 

A. performance problems may exist that are unknown within the organization but that 
frustrate customers. 

B. customers are more aware of the overall effectiveness of operations than those within 
the organization. 

C. most training and development needs can be identified in this way. 

D. customer complaints are important. 

24. A good policy when conducting needs surveys is to 

A. concentrate on concrete responses and avoid eliciting attitudes, motives, and other 
subjective responses. 

B. survey three levels: the target group, their supervisors, and their subordinates. 

C. concentrate only on the target group in order to increase objectivity. 

D. carefully avoid placing constraints on how the survey results will be used before the 
survey is carried out. 
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25. Organizational audits reveal training needs by 

A. using surveys of perceived training needs. 

B. exposing performance deficiencies that show up in records of the results of operations. 

C. eliciting opinions concerning future training needs from a panel of informed managers 
within the organization. 

D. establishing a system of assessment methods that do not rely on existing records. 

26. A goal of the economic aspects of a performance audit is to 

A. decide whether economic benefits justify efforts to improve specific performances. 

B. substitute human behavior measures for economic measures. 

C. reduce the extent to which training is based on economic factors. 

D. provide accurate estimates of training costs. 

27. When a manufacturer had a product quality problem, studies showed that machine operators 
were not able to make needed adjustments nor were they aware of quality- control results. 
Using the performance model, managers would probably conclude that 

A. a training program would be sufficient to solve the problem. 

B. employees needed training, quality-control results, and knowledge of concern about 
quality. 

C. quality standards would have to be changed. 

D. the problem could be solved most easily by replacing the workers. 

28. Improving employee job performance is most likely to have a beneficial result when 

A. the contributions of different types and levels of employees cannot be determined. 

B. both the best and worst performers are very good. 

C. the best performer is currently much better than the worst performer. 

D. the best performer and the worst performer are currently nearly equal in performance. 

29. An organization's plans for the development and use of human resources should 

A. increase productivity and profitability by concentrating on needs for specific skills and 
abilities. 

B. avoid the loss of efficiency resulting from attempts to interrelate individual and 
organizational goals. 

C. develop goals which combine meeting the organization's needs with efforts to promote 
individual growth. 

D. define individual growth goals only in terms of organizational needs. 

30. The most important requirements for bringing about planned and systematic organizational 
change are 

A. numerous personal conflicts within the organization to justify change. 

B. the recognition of felt needs and the support of influential people. 

C emotional equality and fairness currently existing in the organization. 

D. budgets and personnel exclusively devoted to change. 
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31 . The Johari window, when used in career planning, rests on the assumption that 

A. organizational goals and individual goals are ultimately incompatible. 

B. manpower planning is a more important function than career planning. 

C. open two-way communication enhances interpersonal relationships. 

D. communication of information between the organization and individuals is the least 
important aspect of career planning. 

32. Job posting 

A. is not recommended in organizations with formal career planning programs. 

B is usually desired by management but resisted by employees. 

C. is an effective way to give employees control of their career planning. 

D. gives the organization full control over who will succeed to each position. 

33. The most effective training programs are based on 

A. job-relevant behaviors. 

B. general principles of work effectiveness. 

C. behaviors that are not specific to a particular job. 

D. a content analysis. 

34. In the early analysis of a proposed instructional program, the most critical step is 

A. establishing the body of information trainees will be required to learn. 

B. developing a task list of the behaviors desired. 

C. deciding on the methods and techniques to be used by instructors. 

D. assessing the training potential of proposed trainees. 

35. When it has been decided to measure the trainees' level before they enter a training 
program, the goal should be to 

A. sample the entire range of aptitudes that apply to the trainees' positions. 

B. measure the extent to which trainees already possess the knowledge and skills to be 
trained. 

C. develop a measure that will accurately predict the instructional criteria. 

D. obtain an objective measure of potential ability. 

36. A behavioral objective is a 

A. description of the overall goals of a training system. 

B. test used at the beginning and end of instruction to measure improvement. 

C. description of the desired behavioral action, performance conditions, and standards of 
attainment. 

D. description of the behaviors required of instructors to put the instructional system into 
practice. 




32 . 



305 



Page 7 of 7 



37. Who has the primary responsibility for determining what changes in job performance 
should occur? 

A. Trainer 

B. Trainees 

C. Line organization 

D. External training resource 

38. The best way for a trainer to deal with the identification of training needs is to 

A. periodically circulate check-off lists of courses available. 

B. choose educationally sound courses and locate problem areas in the organization they 
will help solve. 

C. systematically and continually collect data to help identify problem areas. 

D. wait for line managers to initiate training requests because these requests reflect true 
problem areas. 

39. Tailoring programs from external resources to specific local needs is easiest 

A. when training needs and course objectives are defined behaviorally. 

B. for traditional courses in which objectives are not specifically defined. 

C. with prepackaged media presentations. 

D. for computer-aided instruction. 

40. In deciding on class size for programs provided by external resources, 

A. it is often necessary to compromise between the small class size desired by line 
managers and the large classes desired by the external resource. 

B. facilities and equipment should not be allowed to limit class size. 

C. classes should be as large as physically possible so all trainees can be accommodated in 
the fewest sessions. 

D. it is often necessary to compromise between the large class size desired by line 
managers and the small classes desired by the external resource. 
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Human Resource Development Department 
Survey of Human Resource Development 



The University of Tennessee 
Final Exam 



Name: SSN: 

Directions: Select any four (4) of the following nine (9) items and answer them completely and 

concisely. Submit your answers along with this sheet at the end of the class session. 

1 . Describe and provide an example of the following: 

A. Formal education 

B. Nonformal training 

C. Informal learning 

2. Explain what a Needs Assessment is and describe what purposes it serves. 

3. The most important result of performance-based training is said to be a measurable 
improvement in an individual’s contribution to workplace/organizational goals. List four 
typical workplace goals and briefly indicate how they justify investing private resources in 
training. 

4. Just prior to WWI, Frederick Taylor introduced scientific management as a way to reduce 
costs. Demonstrate your depth of knowledge by describing, in a concise way, what Taylor 
meant by the phrase scientific management. 

5. Identify five alternative interventions that should be considered before prescribing training 
provide a concise justification for each. 

6. Research found that instructional text followed by post-text questioning resulted in 
performance as effective as programmed instruction. Within the context of the information 
sheet — Research on Preparing Self-Instructional Materials, react to this finding. 

7. Privately-funded training is appropriate when there is a deficiency in knowledge and skills. 
This can happen when (a) new workers are hired, (b) new equipment is purchased, (c) new 
jobs are created, (d) jobs are to be performed in new ways, or (e) jobs are being performed in 
deficient ways by incumbent workers. Publicly-funded training is appropriate when what 
situations exist? 

8. Based on your study of textbook chapter 1, Determining the Market Demand for Skilled 
Workers, and the module, Determining the Demand for Skilled Workers, identify and 
describe concisely the salient characteristics of any two (2) of the five labor market signaling 
approaches. Include a word table showing their advantages and disadvantages. 

9. The practice of grouping students together in a classroom took hold during the Industrial 
Revolution. While group instruction made mass education and training economically 
and administratively feasible, it was at the expense of the individualized tailoring of 
instruction provided by one instructor working with one student. Individualized systems of 
instruction can be viewed as attempts to recapture the effectiveness of individual 
tutoring without the expense. Identify and describe two approaches to individualized 
instruction that proved successful. 



APPENDIX A 



Course Pre-test Answers 



Item no.’s and answers 



1. D 

2. A 

3. B 

4. B 

5. B 

6. C 

7. B 

8. C 



Item no.’s and answers 



9. C 

10. D 

11. A 

12. D 

13. D 

14. B 

15. B 

16. D 
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APPENDIX B 



Progress Test Answers 



Progress test no.’s 




Item no.’s and answers 




1 


1. D 


2. A 


3. 


C 


4. D 


2 


1. B 


2. C 


3. 


A 


4. B 


3 


1. D 


2. B 


3. 


A 


4. C 


4 


1. C 


2. D 


3. 


A 


4. C 


5 


1. D 


2. A 


3. 


D 


4. B 


6 


1. D 


2. A 


3. 


B 


4. A 


7 


1. B 


2. A 


3. 


D 


4. B 


8 


1. A 


2. B 


3. 


C 


4. C 


9 


1. B 


2. D 


3. 


C 


4. D 


10 


1. B 


2. C 


3. 


A 


4. D 


11 


1. C 


2. C 


3. 


D 
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APPENDIX C 



Course Final Exam (Post-test) Answers 



Item no.’s and answers 

1. D 

2. A 

3. A 

4. A 
5 C 

6. D 

7. C 

8. D 

9. A 

10. D 

11. C 

12. D 

13. A 

14. B 

15. B 

16. D 

17. C 

18. C 

19. B 

20. C 



Item no.’s and answers 

21. C 

22. D 

23. A 

24. B 

25. B 

26. A 

27. B 

28. C 

29. C 

30. B 

31. C 

32. C 

33. A 

34. B 

35. B 

36. C 

37. C 

38. C 

39. A 

40. D 
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